Asynchronous Data - Drupalcon DC

05 Mar 2009
Posted by jcfiala

Handling asynchronous Data with drupal

link to notes at

Technical - concepts then code - joy!

Josh_k giving the talk. - drupal-dojo cofounder
Old timer - user/3313

wants to help us prevent his mistakes.
Asynchronous data is the hotness - ajax/ahah, web 2.0
escape the page - become a platform/application

Mission: async_demo.module
Provide a block of links to non-promoted posts - links clicked will cause data to appear at top of /node

Booting up:
* Build Object-Level Data into the DOM class, title, id
* Use drupal.settings for global Info
* Keep your HTML Compliant
* Degrade as Gracefully as Possible.

(Example of embedding data as class/title/link/id)

* Hook to DOM elements w/jQuery
* Better than $(document).ready()
* Override friendly
* Re-attachable
* Contextual

doing node/edit in thickbox, fieldset does not activate. Can re-run the behaviors.
can target behaviors inside of context.

Drupal.behaviors.asyncTeaser = function (context) {
$("a.async-teaser-show", context).bind("click", function() {
var params = $(this0.attr("id").split("-");
return false;

function asyncFrontPagetease(nid) {
var callback = Drupal.settings.basePath + "async_teaser/" + nid;
$.get(callback, function(data) {
$("#squeeze .clear-block:first")prepend(data); //<-- garland specific!
$("#node-" + nid).hide().fadeIn(500);

Difficult to abstract things too much - basically to use AJAX you need to integrate the theme with your ajax, so things will hang together well.

AJAX Menu Calback
* Implement hook_menu() as usual
* Finish with print() instead of return()
* For structured data use drupal_json() (or maybe theme_ahah?)
* be aware of other outputs - e.g. devel (We've had that problem before, where devel was dumping query log, it screws up ajax.)

(Example of a ajax request that returns a node for display)

scaling - 10s of 1000s of requests

* "Live" means up-to-the-second
* Each xHTML request is potentially a full drupal boostrap
* Full drupal is (relatively) expensive.
* Design your flow accordingly

* Earlier drupal bootstrap - php file that calls some bootstrap functions, only what's needed
* dedictaed live.php handler
* static filestore
* Memcached

boostrap is good -
* DRUPAL_DATABASE allows you to read from DB w/ less drupal
* Utilize your variables, db connections, etc
* No access control or other functions
* about 10x faster than full drupal bootstrap
- not fast enough for their high request rate - still hitting database which is performance chocke point.

Static files are Better -
writing to a file, and having the javascript read that file accordingly.
* apache is cheap for performance
* easy to write json to some place in /files
* easy to tell jquery where to look w/ Drupal.settings()
* easy for jquery to grab/parse/act

probably issues with clustered application servers, lots of read/writes, up to the second, or if you read in the middle of an update. Garbage collection - lots of files to clog up filesystem

Memcached = En Fuego
* Shared cloud between many servers - 40-50 application servers even.
* Legendary speed :) Very fast
* Dedicated PHP handler = low overhead - ie, skip boostrap, have php pull out of cloud and return
* Integrate w/Other Caching Strategies.
"Roll with the Thunder" he says

Drupal Example: Live update
Flexible set of methods for responses
api for status changes and content updates
Implemented for PBS engage Live chat
Full site demo w/ populist at 4:15 pm

Security Practices
- Know what needs securing
not really using drupal access restrictions because no/little boostrap
- Security vs. Scalability
Using tokens on links and the like.
* Always use full drupal for writes *
session ids and tokens can help

AJAX Form submissions
Raw power for developing asynchronous stuff

"The Holy Grail"
FormAPI / Standard #attributes / Submits to same URL
DOM/Behaviors/settings, useful defaults, classes, drupal_add_js, custom handlers
AJAX-Submit Form, Client-side Validation, Dynamic Submit handler

currently available contribs:
ajax_submit.module - developer's module, handles submissions
ajax.module -
ahah_family (e.g. ahah_response)

AJAX Submit:
attach via #ajaxsubmit form api attribute
customizeable js callback functionality
Handles errors and previews

Code example on ajax submit.
put blog submit form in sidebar, removing all but title, body, buttons

Lots of thanks to folks.

mention of

question about validation things - didn't really catch it as was distracted.
question about following html specs - doing data changes on get is against html spec. up/down was based on restful things. He thinks that for casting votes, adding friends, easier than doing a POST request. Not too much work to reimplement tokens/post on their own.
question - experience/advice about dynamicly altering froms with ajax/ahah - to get it to work properly with the form caching. Answer - to solve this, sit with drupal6 and cck and pick apart how cck adds additional fields in place. There's a set of functions you can call to let drupal know you have udpated the form, and if you don't do this it doesn't work. In drupal6 the form_build_cache (?) on the server side that you can call to update the form cache that you can use to change the existing cached form.
Some question about iframe and ajax_submit this is apparently pretty standard
How do you create the file that you're polling? Cron? Answer: In context of his app, he has a node for a chat, and when created it creates the text file for the polling, then on a few nodeapi load/submit the loading triggers a write to the state file which lets users see that someone is starting to answer, and then writes the data to the status file, along with the data in another field - all files are written through normal drupal_file_save - good wrapper for writing data to file system.