Squonk Viz Data Centre

Tim Dudgeon 2026-11-03

Squonk

This post describes an important enhancement to Squonk Viz that greatly improves scalability and performance. Data is now stored in a PostgreSQL database running the RDKit cartridge which allows datasets of millions of structures to be used, and provides advanced chemistry functionality like structure search and molecular property predictions.

Early versions of Squonk Viz parsed the data files (typically SD-files or CSV files with SMILES) and that happened in the browser. This performed reasonably well, but clearly had limitations in scale and was not very efficient. Instead of this we now have what we refer to as the “Data Centre” which is essentially a PostgreSQL database with each dataset being a table. Each Squonk project gets its own database schema keeping the data from different projects isolated.

This approach has a number of important benefits

1. file loading becomes a one-time import operation. Once done the data is instantly accessible and there is no need to re-parse the data
2. you can import local files and files from URLs as well as files from your Squonk project
3. operations such as filters, joins and groupings are done by in the database (which is already highly optimised for such operations) rather than a brute force row-by-row analysis in the browser
4. structures can be indexed with the RDKit cartridge providing rapid substructure and similarity search, rather than brute force row-by-row analysis in the browser
5. molecular properties can be calculated/predicted and added to the dataset using the RDKit cartridge
6. indexes can be added to speed up operations like queries

Indexing the structures is optional. It’s only needed if you want to use structure search.

Here is an example of using structure search:

And here is an example of calculating molecular properties:

Here are some sample timings for a number of important operations with various sizes of data. Import and Index are one-time operations - once done they don’t need repeating. The search times are for substructure search and are an average of 3 searches, not including the first search which takes a little longer as the index is being loaded into memory.

The search timing for the biggest set is not actually optimal as the hardware used for the database is quite modest. More memory (especially shared_buffers), faster I/O and more CPU cores would make it a bit faster, but even as it is it’s pretty respectable.

You can access Squonk here.