My GSOC proposal for SahanaPy - HTML/JS based reusable frontend for S3XRC

1. Personal Details

• Name- Abhishek Mishra
• Communication details -
  Email - ideamonk@gmail.com
  Freenode IRC Nickname - ideamonk
  Google talk - ideamonk
  Skype - ideamonk
  Twitter - ideamonk
  Blog - http://blog.ideamonk.in
• I am from a city called Patna in Bihar, India (+5:30 GMT). Currently I am pursuing a B.Tech (a 4 year course) in Amrita School of Engineering, Bangalore. I spend most of my day working on my side projects and cooking new ideas in mind when free from college. In free time and holidays I also work as a freelance web developer with my friends - http://madetokill.com
• Resume - http://ideamonk.in/abhishek_mishra_resume.txt

Exposure to similar technologies and/or FOSS in general.

I started relying heavily on open source projects to learn from and use them ever since I switched to Linux about 1.5 years ago. I started learning Python about 1 year ago and it has in a way become my primary language to implement anything that comes to my mind. I have worked on SahanaPy in the past (the current steel theme, custom error pages, a bit of ExtJS work) and am quite familiar as to how it works. Interestingly SahanaPy is the first Open Source project I've contributed to.

Some thoughts over Free/Open Source Software -
Subtle - as they may look, ignore - as many people may do, but at the end of the day, there's no denying that "Open Source is the best infrastructure you can rely on" to - solve complex problems of the world, help people around the world (the way SahanaPy does!), or even to start your business.
I believe FOSS has saved humanity a lot of time by showing them that "Re-inventing the wheel" is not always the right approach to get things right.
Some of my side projects - http://github.com/ideamonk

“Why would you like to help the Sahana project?”

I was initially introduced to SahanaPy on IRC by a few friends. Later in Dec 2009, I got a chance to participate in SahanaPy workout at FOSS.in. I believe that I could put in my experience in developing Web Applications very well into S3XRC frontend and help developers deal with compound results in an easier, uniform, organized way.
Besides, to the common question that people ask "How would a software help in disaster?", Sahana stands as the best example you could give them, given that it has actually been used at many incidents. The recent support shown to Haiti crisis by Sahana foundation is something that would inspire us to do good at all times. I believe I could too make significant contribution to such a great endeavour.

2. Personal Availability

• Yes I have reviewed the important dates and timeline of GSOC 2010
• Apart from my institute's end-semester exams that fall from April 30th to May 13th, I have no other conflicts with the schedule.
• Will you need to finish your project prior to the end of the GSOC?
  Yes, I would take utmost care to finish it before GSOC ends so that developers could start utilizing it fruitfully by then. Though S3XRC frontend would always have my support post-gsoc too in case any issues/bugs come up.
• I would be available for entire duration of GSOC 2010 starting from May 14th when my exams end.

3. Project Abstract

Currently Sahanapy uses two backends for REST - S3XRC (XML+JSON interface of the restful API) and CRUD (Web2Py crud for html interfaces). Web2Py crud is good at a few things e.g. client side validation, ease of use, but it also has limitations, i.e. It can access only single table at a time.

On current version of SahanaPy this is evident when you take a look at http://127.0.0.1:8000/sahana/hms/hospital/1.xml [1] and compare it to http://127.0.0.1:8000/sahana/hms/hospital/1 [2]

While [1] is generated by S3XRC, it contains nested resources, not only do you get hospital details, but also current status of beds.
On the other hand [2] generated by web2py is not able to do anything worthwhile to this information.

Unfortunately S3XRC has no HTML front-end yet. The backend is able to handle XML and JSON but web2py front-end (SQLTABLE and SQLFORM) are not able to handle the complex resources.

A good way out would be to replace them with new handlers - XRCTABLE and XRCFORM

Which would understand and represent or interpret S3XRC's XML/JSON output appropriately. After that jQuery/ExtJS based web interface would be made which would be more responsive and would be able to take advantage of JSON outputs of S3XRC.

4. Project Plan

This section is to provide the detail of your project proposal. Take as much space as is necessary.

• Project Deliverable -
  
  • The ability to present the complex resources returned by S3XRC to the user.
  • A more responsive jQuery/ExtJS based UI that would change less, and actively load data through AJAX
  • We would be able to save good about of bandwidth, would be able to work in low bandwidth conditions when data gets passed in JSON format.
  • Since aimed to be written in a generic and reusable way, this would be reused in other parts of code too.
  • Besides this work would also reduce the number of requests, which implies quick data entry at times of disaster management work.
• Project Justification -
  
  • The number of useful inter-related data we wish to retrieve and represent would definitely increase in time. This calls for a way to represent then to the user in a RAD fashion.
  • One can never be sure about availability of bandwidth at sites of disaster. At times one might have to work in very low bandwidth conditions. The reduced size and frequency of requests that the S3XRC frontend would bring, will help rescue workers easily operate SahanaPy in such conditions.
• Implementation Plan -
  
  A sample compound resource -
  

  
  <s3xrc success="True" results="1" domain="<span style=" class="J-JK9eJ-PJVNOc">rocksolid</span>"  <span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">url</span>="http://127.0.0.1:8000/<span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">sahana</span>" <span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">latmin</span>="-90.0"  <span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">latmax</span>="90.0"  <span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">lonmin</span>="-180.0" <span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">lonmax</span>="180.0">
    <resource name="<span style=" class="J-JK9eJ-PJVNOc">hms</span>_hospital"  <span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">uuid</span>="eb903352-0a6c-4d3c-bb8c-f18f6b7a3179"  created_on="2010-04-02  15:14:07" modified_by="<span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">Abhishek</span> <span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">Mishra</span>" created_by="<span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">Abhishek</span> <span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">Mishra</span>"  modified_on="2010-04-02 15:14:07"  <span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">url</span>="http://127.0.0.1:8000/<span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">sahana</span>/<span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">hms</span>/hospital/1">
      <data field="website"></data>
     [[ snip snip ]]
     <data field="info_source"></data>
     <data field="phone_exchange"></data>
     <resource name="<span style=" class="J-JK9eJ-PJVNOc">hms</span>_<span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">hrequest</span>" <span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">uuid</span>="c9aa43df-11fc-4b63-977f-d56abaefd1cf"   modified_by="<span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">Abhishek</span> <span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">Mishra</span>"  created_by="<span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">Abhishek</span> <span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">Mishra</span>"  created_on="2010-04-02 15:14:58" modified_on="2010-04-02 15:14:58"  <span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">url</span>="http://127.0.0.1:8000/<span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">sahana</span>/<span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">hms</span>/hospital/1/<span style="background: none repeat scroll 0% 0% yellow;" class="J-JK9eJ-PJVNOc">hrequest</span>/1">
        <data field="city"></data>
         [[ snip snip ]]
       </resource>
   </resource>
  </s3xrc>
  

  
  
  ^^ Notice the nested
  • Approach 1 -
    Addition of new view generators for complex resources as shown above, namely XRCTABLE and XRCFORM to SahanaPy. Using these to generate ExtJS code that would do the interface. We already use ExtJS a lot, so this new front-end would be more generic and fast.
    So, this would be an ExtJS based front-end with S3XRC as backend.
  
  
  • Approach 2 -
    Exploiting S3REST pre and post hooks. When one calls a REST controller, the following happens --
    
    
    1. REST parses the original server request into an XRequest object (jr)
    2. XRequest contains hooks to export / import directly in XML or JSON Eg. jr.export_xml()
    3. After parsing a pre-processor is called - which is dynamic, i.e. a controller can set the pre-processor as response.s3.prep
    4. This means we can pre-process the data before REST action is executed, and even change the REST action itself. Eg. s3rest.set_handler ("foo", my_foo_handler)
    5. If the Pre-processor returns False, then the REST action is not taken at all. This means we can manipulate the flow as we wish to.
    6. At last step, a post-processor is called, which receives both the output of the method handler and the XRequest object. This can be set using response.s3.postp
    7. At this stage we can generate the HTML and ExtJS code required by the views.
    8. The view then builds the generated HTML and ExtJS code into the HTML template.
    
    An optimal approach is to have the ExtJS objects pre-loaded, and retrieve data to them as needed. XRequest supports server-side validation and responds in Ext-compliant JSON. This could be very ideal for a full-fledged Ext-Based interface.
  
  Another point to be taken care of while developing this would be to make it re-usable, generic and easy to customize. For eg. An update-data popup for entries listed in a table could be used where-ever update kind of operation needs to be done, and so on.
• Future Options -
  
  • Adding usability support for browsers that don't have javascript support enabled. This could be either done by making the interface fall back to web2py crud or having an alternate non-js interface rendered by s3xrc
• Relevant Experience -
  
  • I have contributed earlier to SahanaPy - https://launchpad.net/sahana/+topcontributors
  • Some of the SahanaPy related work I did -
    
    • creating the current steel theme, - gives me a good understanding of js, css and views used in sahanapy.
    • improving the main and sub menu (created css based crossbrowser compatible menus),
    • javascript based popup for coordinate conversion and, - I've used ExtJS here for coordinate conversion popup.
    • adding custom error pages - involved tweaking routes.py of web2py framework
    • ( http://is.gd/bj82z http://is.gd/bj86d http://is.gd/bj87m )
  • I have worked with MVC architecture in the past ( CodeIgniter, ASP.NET MVC) and even while working for SahanaPy. Recently started learning Web2Py - http://hello-poll.appspot.com/polls
  • I am quite comfortable with Python, jQuery, writing good HTML & CSS and have basic idea of ExtJS.

Other references - http://trac.sahanapy.org/wiki/S3XRC http://is.gd/bj8Tn

5. Project Goals and Timeline

1. Work already undertaken -
   
   • Reading more about S3XRC, S3REST and drafts of S3XRC_2
   • Familiarize with web2py
2. First trimester (26 April - 23 May) - Community bonding
   
   • Low activity till 13th May due to college exams.
   • As S3XRC work is also related to ucore, I would spend good time before I start coding in learning it more indepth.
   • Creating a list of all things that could be made generic with S3XRC frontend. Right now I'm aware of tables and forms, yet inside forms we could provide a generic way to make them interactive as seen in a work by Michael - http://58.137.55.83/adpcdemo/drrpp/project/create
   • Researching some more possible generic widgets that could be implemented in S3XRC frontend and reused later.
   • Collecting feedbacks from developors as to what kind of tasks are restricted due to current web2py crud.
3. Second trimester (24 May - 12 July) - Coding starts
   
   • Basic XRCTABLE 24th May to June 13th
   • ExtJS interface to XRCTABLE to provide on the spot update/add feature - June 14th to June 27th
   • XRCFORM as a replacement to existing SQLFORM - June 28th to July 12th
4. Third trimester (13 July - 9 August) -
   
   • ExtJS interface for XRCFORM elements to reduce number of requests, and to provide inline edits, assistive popups, etc - July 13th to July 20th
   • Weekly implementation of widgets and generic elements decided in First Trimester - July 21st to Aug 7th
   • Rigorous testing and bug fixes - Aug 7th to Aug 9th