IdeaMonk

thoughts, ideas, code and other things...

Tuesday, September 28, 2010

Reason why HazeNet is as good as hosting your website on a friend's adsl

Labels: , , ,

Wednesday, September 22, 2010

Speaking at PyCon this weekend :)

Here are the slides for talk at the upcoming PyCon India 2010. See you this weekend.

See you there!

Labels: ,

Saturday, September 18, 2010

Setting up Lighttpd, PhpMyAdmin, etc

Recently from a tipoff from Suryajith, I decided to try migrate to lighttpd and test it out. I was looking for some more reduction in active ram usage, so that I could also replicate the same on our 256mb vps. Here is how I went about it -

  1. stop apache 2
    # /etc/init.d/apache2 stop


  2. remove all related packages
    # apt-get remove apache2  apache2-utils apache2-common --purge


  3. Get lighttpd, mysql, php working with these instructions for Debian. Debian provides FastCGI enabled lighttpd. How awesome!

  4. Now you would need to remap configs if you had virtual hosts, etc. Take a look at /etc/lighttpd/lighttpd.conf for it.
    server.document-root = "/path/to/master/doc/root/"
    As for subdomains, I have the following -
    $HTTP["host"] =~ "blog.ideamonk.com" {
    server.document-root = "/path/to/sites/blog.ideamonk.com/"
    }

    $HTTP["host"] =~ "www.ideamonk.com" {
    server.document-root = "/path/to/sites/ideamonk.com/"
    }


  5. Now get phpmyadmin, if you had it setup with apache2, better remove it with a purge and reinstall. Set it to work with lighttpd when the reconfiguration menu comes up during installation.

  6. Add a url alias for it to the lighttpd config. On Debian, it usually gets installed under /usr/share/phpmyadmin/ . I've setup the following alias globally in my config, so appending /pma/ to any subdomain pops up phpmyadmin. Its good till the while I've just myself on the machine, once you've got may websites on the same box, be prepared to explain why they can't use "/pma/" in their url, or better shift this alias to your support website.Global alias-
    alias.url += ( "/pma/" => "/usr/share/phpmyadmin/" )

    PhpMyAdmin alias attached to a subdomain -
    $HTTP["host"] =~ "admin.myfooservice.com" {
    alias.url += ( "/pma/" => "/usr/share/phpmyadmin/" )
    server.document-root = "/path/to/admin.myfooservice.com/"
    }


  7. Now if phpmyadmin doesn't show up, you probably have permission issues, while this works smoothly on apache2, I had to assign group id of 'www-data' to all files under /usr/share/phpmyadmin/
     $ sudo chown root:www-data -R /usr/share/phpmyadmin/


  8. Now you would notice, almost same or more amount of ram usage that your old php-mysql-apache2 setup. Don't be disappointed, Its the php5-cgi processes and their children hogging up more. And so might be the mysql server. You need to limit the php5-cgi processes. Make modification for this in lighttpd.conf, here's my config -
    fastcgi.server = ( ".php" => ((
    "bin-path" => "/usr/bin/php5-cgi",
    "socket" => "/tmp/php.socket",
    "max-procs" => 1,
    "idle-timeout" => 20,
    "bin-environment" => (
    "PHP_FCGI_CHILDREN" => "4",
    "PHP_FCGI_MAX_REQUESTS" => "10000"
    ),
    "bin-copy-environment" => (
    "PATH", "SHELL", "USER"
    ),
    "broken-scriptfilename" => "enable"
    )))


  9. As for tuning up mysql, have a look at the linuxweblog tutorial. Here is my configuration.


End results -
top - 05:18:13 up 5 days,  2:55,  1 user,  load average: 0.07, 0.08, 0.07
Tasks: 20 total, 1 running, 19 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.2%us, 0.0%sy, 0.0%ni, 99.8%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 1048576k total, 107480k used, 941096k free, 0k buffers
Swap: 0k total, 0k used, 0k free, 0k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
19711 root 15 0 6800 2516 2064 S 0 0.2 0:00.01 sshd
1 root 15 0 1980 584 552 S 0 0.1 0:01.01 init
5435 www-data 18 0 8216 2148 924 S 0 0.2 0:00.02 lighttpd
5437 www-data 23 0 34428 10m 7288 S 0 1.0 0:00.07 php5-cgi
5442 www-data 15 0 50416 23m 4740 S 0 2.3 0:12.70 php5-cgi
5443 www-data 16 0 50368 22m 3984 S 0 2.2 0:00.70 php5-cgi
13684 ideamonk 15 0 4344 2940 1300 S 0 0.3 0:00.16 bash
13945 root 15 0 1692 580 484 S 0 0.1 0:02.19 syslogd
16193 root 18 0 2352 580 576 S 0 0.1 0:00.00 xinetd
18118 root 15 0 5272 1028 676 S 0 0.1 0:00.24 sshd
19702 ideamonk 15 0 2256 1080 880 R 0 0.1 0:00.00 top
19715 sshd 15 0 6616 1352 928 S 0 0.1 0:00.00 sshd
24555 root 15 0 8264 684 520 S 0 0.1 0:02.13 sendmail-mta
25719 root 15 0 2036 756 692 S 0 0.1 0:00.15 cron
28000 root 16 0 8016 2648 2188 S 0 0.3 0:00.01 sshd
28104 ideamonk 15 0 8168 1528 1052 S 0 0.1 0:00.20 sshd
28112 ideamonk 18 0 3316 1860 1248 S 0 0.2 0:00.00 bash
32444 root 20 0 2488 1068 956 S 0 0.1 0:00.00 mysqld_safe
32506 mysql 15 0 45548 4848 3068 S 0 0.5 0:01.32 mysqld
32507 root 23 0 1628 460 456 S 0 0.0 0:00.00 logger

About 90 - 107M average ram consumption on my vps :D
You can notice php5-cgi threads taking up 10-25M, without them you can have a static page server with around ~50M ram usage!

Update - Trying further more I realized that I have too many unused php5 modules installed, for instance php5-imap, php5-snmp, etc. I decided to remove then all and let phpmyadmin dependencies pull php5 and bare minimum modules. Now the php5-cgi children take from 5M to 16M with idle ram usage around 60-70M :)

Saturday, September 11, 2010

Getting Python + OS X together

Since a week I’ve been looking for ways to create Mac apps using Python. Here are a few important tutorials/articles which got me my setup -

Many tutorials show up CocoaPython option in XCode’s ‘create new project’ wizard. It wasn’t there in XCode 3.2.3, so I made myself believe that I need to get something, and so began my exploration.

At first I decided that it would be cool to be able to write a pygame app and turn it into a portable binary -

  • Thanks to kontinuity setting up PyObjC on Python 2.6.5 is a cakewalk
  • Now you need pygame, which is another recipe in itself. Grab the pdf guide here. Clear instructions that work
  • Py2App would fail looking for
    “/python2.6.5/lib/python2.6/site-packages/py2app-0.5.3-py2.6.egg/py2app/apptemplate/prebuilt/main-x86_64

    Just copy the main-fat file in the same path as main-x86_64

  • Test py2app over a small pygame example/script. Now the app would fail to load up. Have a look at the ‘Console’ -
    If using setuptools, just set the argv_emulation under OPTIONS as False to pass through.
    Also don’t forget to add ‘zlib’ to includes in case console shows errors related to a zlib import.
  • Now you have your working pygame+py2app setup. Also if you’re looking for Physics engines, I would recommend you go for PyBox2D. Though PyBox2D’s documentation is sparse and examples lacking, it is something that works compatibly with py2app.
    I also tried pymunk, though it builds up and works fine, but the py2app generated binary always dies out in a segmentation fault. Pretty unreliable for py2app as observed on OSX 10.6.4. Also you would have to put in libchipmunk.dylib manually into app archive after every build.

Now thats cool in case you’re an indie game-dev aligned python charmer with a mac.

For those of you looking for XCode+PyObjC development, the good news is that Python2.5 and PyObjC 2.0 have been bundled in your OS X installation ever since 10.5. All you need to do is to follow what Ioanna has to say about getting the XCode templates in. But remember this would use up python26-apple instead of your goody macports versions and would have its own set of issues along (as in with getting the 3rd party modules up and running)

I couldn’t get all of it working on XCode 3.2.3, OS X 10.6.4, if you too don’t see ‘Python’ under File Templates in XCode, when say you’re trying to add a class through the gui, just follow these -
Goto ~/Library/Application\ Support/Developer/Shared/Xcode/File\ Templates/ where you have copied the File Templates. Create a folder named Python here and move all the “Python NS…” folders inside it.

Happy coding!

Labels: , , , , , ,