12.29.2010
whiskey db backend
After playing around with it and reading the specs, yaml is not the right tool for the job. It's too hard to have multiple attributes for the same key (ie, colours: grey, blue, green) and even harder to parse that back out and search it. Some sort of nosql db seems to be what I'm looking for, and mongodb seems to be the friendlier of the python-supported nosql dbs out there. So... scratch the yaml stuff (until it comes time for config files, that's still a good idea), mongodb is going to be the db backend. And I should have some working, prealpha code going up on launchpad soon. No promises though, my work hours have expanded (yay money, boo exhaustion) so it's tough to get anything done but work, chugging coffee, and sleeping. But I'm trying. :)
9.26.2010
Whiskey
Now I'm working on a desktop-agnostic wallpaper manager. I jokingly called it 'whiskey' one night in a chat, and the name kinda stuck. Anyhoo, it walks through the user's home directory, looking for jpegs, gifs, pngs, and xcfs, looks at the size to determine if it's a wallpaper by its size (4:3 and 16:9 sizes are supported), and if it's not a standard size, it asks the user if they want it added to the database. The database is written in YAML. What will be stored in the database is the filename (full path), format, size, aspect ratio, colours, and design details. The colours and design details will be user-supplied. The gui will be done with pyglet. The image processing library I'm using to parse the metadata and will be displaying the preview in the gui is PIL (Python Image Library). Though, PIL's docs say the function to display the image is really inefficient, so if it turns out to suck too badly, I'll find something else to handle that.
I'm thinking to handle tiling, I'll grab the screen resolution (somehow) and use PIL to paste the image over and over into a bigger image the size of the screen, then save it as something like "foo.tiled.png" or so.
I'm still not sure exactly how I'm going to actually set the wallpaper, whether I'll shell out to fbsetbg/esetbg/etc., or write my own routine to handle it.
The way to look through the wallpapers is a slideshow. There'll be a 'set' button, and a delete button that'll trigger a menu to either simply delete the file from the database, or from the system.
So far that's all I have planned out. Which I suppose kinda makes sense, since all I have coded so far is the function to walk down the home directory, and the function to read the metadata prior to writing it to the database. And I have a sketch of the UI (not my work, my fiance is my design person).
I'm thinking to handle tiling, I'll grab the screen resolution (somehow) and use PIL to paste the image over and over into a bigger image the size of the screen, then save it as something like "foo.tiled.png" or so.
I'm still not sure exactly how I'm going to actually set the wallpaper, whether I'll shell out to fbsetbg/esetbg/etc., or write my own routine to handle it.
The way to look through the wallpapers is a slideshow. There'll be a 'set' button, and a delete button that'll trigger a menu to either simply delete the file from the database, or from the system.
So far that's all I have planned out. Which I suppose kinda makes sense, since all I have coded so far is the function to walk down the home directory, and the function to read the metadata prior to writing it to the database. And I have a sketch of the UI (not my work, my fiance is my design person).
8.13.2010
Version control
Rather than using git (or cvs or svn) we're going with a bazaar repo/server. It seems to be the best documented, easiest to use system.
I've used cvs in the past, and there's a significant learning curve and little documentation outside of the (typically terse) manpage. trash80 has used svn and hates it. I found the git wiki with a "git for the lazy" guide, and it's still not the clearest docs in the universe.
This: http://doc.bazaar.canonical.com/explorer/en/index.html just seems good, and it's available on both ubuntu (trash80's distro of choice) and arch (my distro of choice).
I've used cvs in the past, and there's a significant learning curve and little documentation outside of the (typically terse) manpage. trash80 has used svn and hates it. I found the git wiki with a "git for the lazy" guide, and it's still not the clearest docs in the universe.
This: http://doc.bazaar.canonical.com/explorer/en/index.html just seems good, and it's available on both ubuntu (trash80's distro of choice) and arch (my distro of choice).
7.30.2010
Manifesto Time
OK, I'm bored with the kernel thing right now, and we need something to install before I really get into it. So I'll be moving on to hacking pacman to use a git repository and rewriting package dependencies. But right now it's time to clear my head, step back, and look at the bigger picture-- so on to the promised jet-fu manifesto.
The over-all aim of jet-fu is to be a small server. This means:
> Only dependencies for packages will be the bare minimum to run the package.
- trash80 described it better than I ever could in his first post, but basically just because a package *could* need something to enhance its features, doesn't mean it should automatically be installed. If the server admin wants it, (s)he can grab it themselves. Optional dependencies will be listed in at least one place for reference so they can be installed as well if the admin so desires. I like that pacman already simply lists optional dependencies on install, I wouldn't change that feature, others may disagree though. If mainstream pacman changes this behaviour, we may fork. Another possible place to list dependencies would be a wiki or something.
> Small, customized kernel from the moment of install.
- Rather than having to roll your own kernel to clear up all the cruft and extra drivers a distro kernel installs (and which the kernel trudges through on boot each time to see which ones need to be loaded), the installer will only put in the base kernel and the drivers needed to run the exact hardware/software that's configured at the moment. Additional modules will be available online (on the package server?), and can be downloaded on-the-fly when they need to be loaded, and then become persistent in the system.
> No gui, because servers don't need guis.
- Sounds simple enough. No X or anything like that available, just command-line access or a web interface. Something along the lines of webmin, but better. Since this is already geared towards LAMP servers, all the tools are there and running, just log into the admin interface and do what you need to do.
Obviously, right now we're basing the distro on arch, to some degree. They have the most featureful package manager that still runs on a distro that's close to what we want to achieve. I realize the apt suite and *rpm* are at least as featureful as pacman, but debian has gotten bloated over the years, and rpm-based distros have always tended to be bloated.
Some ideas we're playing around with that may or may not make it to release are: git repo server, ditching HAL (since it seems only certain gui apps like thunar use it now) and writing our own udev. Again, these are subject to change. I may also have forgotten some things, so there may be a manifesto part 2 in the future.
One last thing, all tools we write will be in python. We're going with python 2.6 as our standard version, with our code geared to 3.0, then upgrading to 3.0 when it becomes more mainstream.
The over-all aim of jet-fu is to be a small server. This means:
> Only dependencies for packages will be the bare minimum to run the package.
- trash80 described it better than I ever could in his first post, but basically just because a package *could* need something to enhance its features, doesn't mean it should automatically be installed. If the server admin wants it, (s)he can grab it themselves. Optional dependencies will be listed in at least one place for reference so they can be installed as well if the admin so desires. I like that pacman already simply lists optional dependencies on install, I wouldn't change that feature, others may disagree though. If mainstream pacman changes this behaviour, we may fork. Another possible place to list dependencies would be a wiki or something.
> Small, customized kernel from the moment of install.
- Rather than having to roll your own kernel to clear up all the cruft and extra drivers a distro kernel installs (and which the kernel trudges through on boot each time to see which ones need to be loaded), the installer will only put in the base kernel and the drivers needed to run the exact hardware/software that's configured at the moment. Additional modules will be available online (on the package server?), and can be downloaded on-the-fly when they need to be loaded, and then become persistent in the system.
> No gui, because servers don't need guis.
- Sounds simple enough. No X or anything like that available, just command-line access or a web interface. Something along the lines of webmin, but better. Since this is already geared towards LAMP servers, all the tools are there and running, just log into the admin interface and do what you need to do.
Obviously, right now we're basing the distro on arch, to some degree. They have the most featureful package manager that still runs on a distro that's close to what we want to achieve. I realize the apt suite and *rpm* are at least as featureful as pacman, but debian has gotten bloated over the years, and rpm-based distros have always tended to be bloated.
Some ideas we're playing around with that may or may not make it to release are: git repo server, ditching HAL (since it seems only certain gui apps like thunar use it now) and writing our own udev. Again, these are subject to change. I may also have forgotten some things, so there may be a manifesto part 2 in the future.
One last thing, all tools we write will be in python. We're going with python 2.6 as our standard version, with our code geared to 3.0, then upgrading to 3.0 when it becomes more mainstream.
7.27.2010
kernel scripts 2
OK, after digging around, I've learned some things. modprobe.conf doesn't list the modules to be loaded (did it at one point? I don't know. But it doesn't now), just the rules for things like naming/aliasing modules, local dependencies, substituting one module for another, etc. If I'm reading the docs right, that is. It seems mostly for manually running modprobe, and tweaking its behaviour so that in general, whether modules are automatically or manually loaded, it can be made easier to understand; or to compensate for local fringe case behaviour. Either way, not really helpful.
I'd thought in the past the kernel had a list of all modules it loaded on boot. Either this isn't the case anymore, or I've been mistaken all these years. *shrug* Whatever. The kernel does the loading, talks to udev about it, and on the upside, it doesn't say anything about udev freaking out and crashing the system if the kernel can't find the module it wants. But, this could make my life a little harder in the kernel paring script if it turns out modinst needs to interface with udev to work properly. Maybe there's a way to wedge modinst in between the kernel, modprobe, and udev? This is sounding harder and harder to do. It can't be impossible, but I should probably apologize in advance to my fiancee for the frustrated screaming he'll have to listen to. ;)
Arch has an option in rc.conf to disable udev autoloading on boot, and specify which modules to load. I'm getting ready to play around with that in a vbox image. One other thing I have going for me is, servers usually don't have a lot of hotplugging going on, or if they do (ie, scsi disks) it's with identical parts. Unlike a desktop with random devices getting plugged in and unplugged, which the kernel has to cope with.
Trash80's idea of writing our own hal and udev (pykal and pydev) is sounding really good right now.
I'd thought in the past the kernel had a list of all modules it loaded on boot. Either this isn't the case anymore, or I've been mistaken all these years. *shrug* Whatever. The kernel does the loading, talks to udev about it, and on the upside, it doesn't say anything about udev freaking out and crashing the system if the kernel can't find the module it wants. But, this could make my life a little harder in the kernel paring script if it turns out modinst needs to interface with udev to work properly. Maybe there's a way to wedge modinst in between the kernel, modprobe, and udev? This is sounding harder and harder to do. It can't be impossible, but I should probably apologize in advance to my fiancee for the frustrated screaming he'll have to listen to. ;)
Arch has an option in rc.conf to disable udev autoloading on boot, and specify which modules to load. I'm getting ready to play around with that in a vbox image. One other thing I have going for me is, servers usually don't have a lot of hotplugging going on, or if they do (ie, scsi disks) it's with identical parts. Unlike a desktop with random devices getting plugged in and unplugged, which the kernel has to cope with.
Trash80's idea of writing our own hal and udev (pykal and pydev) is sounding really good right now.
7.25.2010
Kernel scripts
Right now the main thing that has my attention is the kernel paring (ironically, one of the minor details and one of the last things that actually needs to be done). It's a 2-part problem, preinstall (or more precisely, during install) and post-install.
Jet-fu's vanilla kernel will be as modular as possible to allow it to be as small as possible.
For preinstall, the livecd boots, the kernel does its magical hardware detection (I still have to delve more into kernel hardware detection, so it's still magical ;) ). When the installer is run, it'll run lsmod, pull the output, clean it up so just the module names are written to modules.conf, write modules.conf. Then use modules.conf to install just the modules that are actually needed.
For post-install, I actually have some pseudocode. That script will be called modinst, and it'll be used in place of modprobe (more specifically, insmod). So when modinst foo_mod is run:
It triggers a script to grab module from server and put it into the modules dir and then run modprobe to install it.
So far this looks good to me, I welcome comments on whether this actually sounds like it'll do what i want it to do, or improvements.
Jet-fu's vanilla kernel will be as modular as possible to allow it to be as small as possible.
For preinstall, the livecd boots, the kernel does its magical hardware detection (I still have to delve more into kernel hardware detection, so it's still magical ;) ). When the installer is run, it'll run lsmod, pull the output, clean it up so just the module names are written to modules.conf, write modules.conf. Then use modules.conf to install just the modules that are actually needed.
For post-install, I actually have some pseudocode. That script will be called modinst, and it'll be used in place of modprobe (more specifically, insmod). So when modinst foo_mod is run:
cd /path/to/modules
wget -nvc ftp://server.for.mod/foo_mod
modprobe foo_modIt triggers a script to grab module from server and put it into the modules dir and then run modprobe to install it.
So far this looks good to me, I welcome comments on whether this actually sounds like it'll do what i want it to do, or improvements.
7.24.2010
first post!!!1!!
Ok, got that out of my system. Hi! Mostly this blog will be about stuff I'm hacking on, commentary on Linux, languages, and other computer tech I feel unusually passionate about, and I'll end up at some point touching on the trials and tribulations of being a starving linux hacker working a dead-end non-IT job. Fun!
Currently I'm working on a really cool experimental distro called Jet-fu (Just Enough To FUnction, among other possible backronyms), relearning Python, getting more into C programming, as well as geting into modding computers. Check out http://wsrogers.blogspot.com/, he's the other half of this crazy distro. :)
In a nutshell, jet-fu aims to be a server distro that installs and runs in the smallest possible space. It will be geared for intermediate-advanced users (ie, sysadmins). Right now my pet projects as part of the groundwork are: writing scripts to pare down the kernel on install and grab modules by ftp as need be; figuring out what other specialized scripts the installer will need and fleshing those out; and working on the package management problem (how, where, and with what).
A full 'manifesto' of the project is coming soon.
As for random life-rants, I'll try to keep those to a minimum. I know my job is not that interesting, and this is mostly about hacking. Hopefully I'll keep that promise....
Currently I'm working on a really cool experimental distro called Jet-fu (Just Enough To FUnction, among other possible backronyms), relearning Python, getting more into C programming, as well as geting into modding computers. Check out http://wsrogers.blogspot.com/, he's the other half of this crazy distro. :)
In a nutshell, jet-fu aims to be a server distro that installs and runs in the smallest possible space. It will be geared for intermediate-advanced users (ie, sysadmins). Right now my pet projects as part of the groundwork are: writing scripts to pare down the kernel on install and grab modules by ftp as need be; figuring out what other specialized scripts the installer will need and fleshing those out; and working on the package management problem (how, where, and with what).
A full 'manifesto' of the project is coming soon.
As for random life-rants, I'll try to keep those to a minimum. I know my job is not that interesting, and this is mostly about hacking. Hopefully I'll keep that promise....
Subscribe to:
Posts (Atom)