Common Lisp SDK for the Datastar Hypermedia Framework

Datastar Common Lisp SDK

================================================================

   |             
  / \            DATASTAR HYPERMEDIA FRAMEWORK
 /___\          
|  D  |          
| C*L |                '(DATASTAR-CL)    
|     |                 

/| | |
| |_ |__| | COMMON LISP SDK |/ /.\ |

================================================================

What is it This is a Common Lisp implementation of the Datastar SDK, following the Datastar Architecture Decision Record (ADR) as closely as possible, making the necessary adjustments for Common Lisp. Focus has been on implementing the core SDK functionality, with additional utilities planned. How to use it Clone the repository somewhere ASDF2 can find it (~~/src/lisp~ is commonly already configured), and use Quicklisp to load it: (ql:quickload "datastar-cl")

Approach The SDK uses CLOS and is based on a sse-generator class, that currently has two subclasses:

hunchentoot-sse-generator: used with the Hunchentoot web server. clack-sse-generator: used with the Clack web application environment.

Both approaches use a stream, although in different ways and initialized differently. For Clack, the work done is at the Lack level, and has been tested with both Hunchentoot and Woo. We are currently using JZON as the JSON parser in two places: read-signals and patch-signals. This is not a hard requirement to remove, but for now it allows immediate use and testing of the library. SSE and Common Lisp webservers When using Clack with Woo, SSE connections are limited by Woo’s worker thread count. Each SSE connection blocks one worker for its entire duration. This is a fundamental architectural constraint - see SSE-WOO-LIMITATIONS.org for details and practical solutions. For applications with many SSE connections, Hunchentoot is recommended (no connection limit), or using a polling approach. By default, Hunchentoot (and Clack+Hunchentoot) uses a push approach, while Clack+Woo defaults to poll. Examples The test/ directory has some basic examples. Additionally, the following projects use it

Data SPICE: uses Datastar-CL for a 2D simulation of the solar system and Cassini-Huygens probe; written in parallel with the SDK to showcase it and also make sure it worked. Horizons JPL API explorer: in the same theme, but a vastly simplified app that queries the Horizons API for solar system data.

The Horizons JPL app is very simple – one single file – and should show the essentials. The Data SPICE application shows SSE streaming. Compression Support for compression was recently added, currently covering all backends but limited to zstd: read COMPRESSION.org for details. Testing The test/ directory contains the code to setup and start the supported servers, Hunchentoot and Clack (with Woo and Hunchentoot backends, others can be added). It’s also a good place to use as an example of how the SDK can be used, especially since there are some differences depending on the backend (Hunchentoot or Clack). SSE has a specific test file that can also serve as an example, and some important edge cases (like what happens when connections are interrupted) are tested as well. License The code follows Datastar’s licence choice, the MIT license.