Stefano Tommesani

The Flash library by Mikal Hart is a wonderful piece of software for improving simple web server apps built on the Arduino: the main problem when designing such apps is that the limited number of RAM available forces the designer either to use an SD card to store the files that are part of the web site, and/or store some HTML fragments in the PROGMEM area, and copy them back to normal RAM to modify and include them in the server HTML packets. The Flash library greatly simplifies the second technique, as the following code fragment highlights:

FLASH_STRING(WebPageHeader, "HTML source code");
Client& client = web_server.get_client();
WebPageHeader.print(client);
 

The FLASH_STRING macro declares a string contained in PROGMEM, so that it does not decrease the available RAM. Streaming that string is as easy as calling the print method and passing the Client connection of the web server as the parameter. Using this technique, dynamic generation of web pages includes both static parts stored in PROGMEM as FLASH_STRINGs (for example, page headers and footers, top menus, etc.) and dynamic parts that are generated by code (for example, tables containing data values coming from attached devices).

Here is video footage from a real-life scene: it's a windy day, so the camera is slightly shaking, and there is a lot of action happening in the road below. This video sequence is useful for the following tests:

• - checking the direction of moving vehicles (there are bikes moving in the wrong direction)
• - classifying moving objects (cars, buses, motorcycles, bikes etc.)
• - rejecting false alarms due to unrelevant movements (the shadow of the trees is costantly moving, but it should not trigger an alarm, while people walking and running should)

You can download this video sequence (running time: 9m 42s) from Dropbox clicking on this link. (if clicking on the link starts a video player, right-click on the link and choose Save Link As.. from the popup menu)

An application of mine uses DirectDraw to draw video frames on multiple screens, and so far the visualization pipeline used a group of off-screen YUV surfaces that, at the end of the process, are drawn into the primary surface. So this is the code that creates the primary surface:

DDSURFACEDESC ddsd;
 ZeroMemory(&ddsd, sizeof(ddsd));
 ddsd.dwSize = sizeof(DDSURFACEDESC);
 ddsd.dwFlags = DDSD_CAPS;
 ddsd.ddsCaps.dwCaps    = DDSCAPS_PRIMARYSURFACE;
 if ((ddrval = lpDD->CreateSurface(&ddsd, &lpDDSPrimary, NULL)) != DD_OK)
 {
 return VISLIB_ERROR;
 }

and this is the code that creates an off-screen YUV surface:

In the AltaPixShare software app, when the user drags a group of images to the target location, the following code iterates over the list of images, and for each image creates the destination file:

for i := 0 to Pred(ActiveImages.Count) do
 begin
 try
 CurrentImage := TProcessedImage(ActiveImages[i]);
 except
 CurrentImage := nil;
 end;
 if Assigned(CurrentImage)
 then begin
 CurrentImage.CreateDestFile();
 end;
 end;

Given that applying a filter to a large image and then writing the output image into a JPEG file is a time-consuming task, this is a clear case for multi-threaded optimization, so that multiple images are processed at the same time on different CPU cores. And using the amazing AsyncCalls library, we can achieve this result with only a few lines of code. These are the steps required to turn this serial code into a multi-threaded one:

Download SimpleDBExplorer project files and executable

Introduction to Amazon SimpleDB

The best description of the Amazon SimpleDB service is given by Amazon itself here:

Amazon SimpleDB is a highly available and flexible non-relational data store that offloads the work of database administration. Developers simply store and query data items via web services requests and Amazon SimpleDB does the rest. Unbound by the strict requirements of a relational database, Amazon SimpleDB is optimized to provide high availability and flexibility, with little or no administrative burden. Behind the scenes, Amazon SimpleDB creates and manages multiple geographically distributed replicas of your data automatically to enable high availability and data durability. The service charges you only for the resources actually consumed in storing your data and serving your requests. You can change your data model on the fly, and data is automatically indexed for you. With Amazon SimpleDB, you can focus on application development without worrying about infrastructure provisioning, high availability, software maintenance, schema and index management, or performance tuning.

Amazon stresses that SimpleDB is optimized for high availability, thaks to distributed replicas of data, and low administrative costs, as there are no schemas and indexes to manage. With SimpleDB, you don't have to define a schema for each database table before you can use it, and you don't have to change that schema before storing your data in a different way. As there is no schema, there are no data types, as all data values are treated as variable length char data. So, if you want to add a new field to an existing database, you just data the new field to the data items that require it, and there is no enforcing that all data items contains the same fields. The drawback of having no schema is that you have to handle formatting and type conversions in your code, and this has a serious impact on queries.

The terminology used in SimpleDB is different from the usual DBMS one, so the following table shows how the different terms are related:

The domain is analogous to a database table, but a domain has no schema, so the only parameter you can set is the name of the domain. All the data stored in a domain has the form of name-value attribute pairs, and each attributes pair belongs to an item, which is similar to a table row. The attribute name is similar to a table column, but each item can have different attribute names, so you can store data with different layouts in the same domain, and add new fields to items without restructuring the table. It is also possible to have, for a given attribute, not just one value, but an array of values, so when you add an attribute name-value pair to an item, you can choose if SimpleDB should replace the existing pair with the same attribute pair, or add the new value to the existing pair.

Queries on SimpleDB are mostly done with a key-value approach, where you retrieve an item and its attributes from the item's name. But you can also use a SQL-style query language to issue queries over the scope of a single domain. When designing queries, you should be aware that all the data stored in SimpleDB is treated as plain string data (be careful with numerical data!), and that all values are automatically indexed.