Versatile GPU-accelerated drawer implementation for OpenSeadragon. The design originates from xOpat viewer, where the basis for this rendering engine was developed.
See it in action and get started using it at https://openseadragon.github.io/.
OpenSeadragon v6.0+ is required to use this renderer. It is a drop-in replacement for the default OpenSeadragon drawer, which means you can use it as a regular OpenSeadragon drawer. Load this renderer after OpenSeadragon and before creating the viewer. It will automatically register itself as a renderer option.
let viewer = OpenSeadragon({
drawer: 'flex-renderer',
drawerOptions: {
'flex-renderer': {
// optional renderer configuration
// debug: true
}
},
// other options like id: ...
});Then, you can use one of built-in (or implement custom) visualization styles by
configuring TiledImages via JSON shader layers. The configuration can happen in two ways:
- handled internally: each Tiled Image will be automatically assigned
identityrendering style, which you can customize by callingviewer.drawer.configureTiledImage(tiledImage, { type: 'identity' });
- handled externally: completely override the renderer output by custom rendering configuration
viewer.drawer.overrideConfigureAll({ 'key1': { type: 'identity', tiledImages: [0], }, 'key2': { type: 'identity', tiledImages: [1], }, 'key3': { type: 'identity', tiledImages: [2], } }, ['key1', 'key3', 'key2']); // we can define custom layer order for rendering
In the first case, TiledImage configurations (like blending mode) are respected. In the second case,
the TiledImage settings are completely overridden by the provided configuration. That means properties
like opacity or blendMode are ignored on the TiledImage level, and read only from the provided JSON.
Shader layer is a definition of how one or multiple tiled images are rendered. They allow you to
provide custom parameters and customize things like what channels you sample from the data.
Except for the type property the golden rule is: don't specify what you don't need.
{
"type": "identity",
"name": "Probability layer",
"visible": "1",
"tiledImages": [0], // indexes of tiled images to sample from
"params": {
"use_gamma": 2.0, //global parameter, apply gamma correction with parameter 2
"use_channel0": "grab", //global parameter, identity shader expects 4 channels - we reorder rgba -> grab
"use_mode": "show", //global parameter, blend mode context for the layer ("show", "blend", or "clip" only)
"use_blend": "add" //global parameter, blend mode for the layer (mask, add, multiply, screen, overlay, etc.)
}With use_mode=show the blending is ignored. With blend, blending is respected, with clip applied only against the previous layer.
Updates to the configuration are generally reflected immediately, if you re-build the program.
When you want to let users to control shader inputs through UI, you need to provide a handler for rendering the UI components. This handler MUST register the component UI to DOM when called.
drawerOptions: {
'flex-renderer':{
htmlHandler: (shaderLayer, shaderConfig) => {
const container = document.getElementById('my-shader-ui-container');
// Create custom layer controls - you can add more HTML controls allowing users to
// control gamma, blending, or even change the shader type. Here we just show shader layer name + checkbox representing
// its visibility (but we do not manage change event and thus users cannot change it). In case of error, we show
// the error message below the checkbox.
// The compulsory step is to include `shaderLayer.htmlControls()` output.
container.insertAdjacentHTML('beforeend',
`<div id="shader-${shaderLayer.id}">
<input type="checkbox" disabled id="enable-layer-${shaderLayer.id}" ${shaderConfig.visible ? 'checked' : ''}><span>${shaderConfig.name || shaderConfig.type}</span>
<div>${shaderLayer.error || ""}</div>
${shaderLayer.htmlControls()}
</div>`);
},
htmlReset: () => {
const container = document.getElementById('my-shader-ui-container');
container.innerHTML = '';
}
}
}UI Components are named arbitrarily (note the reserved use_ prefix for global parameters though).
They can be configured like so:
{
type: 'heatmap',
params: {
'color': '#ff0000', // color to use for the heatmap
}
}Or:
{
type: 'colormap',
params: {
'color': {
'type': 'color',
'default': '#ff0000'
// .. and other properties - depends on the target control type
}
}
}Note that the name of the control in params depends on the shader layer.
Shader layer defines color as a name for UI control:
static get defaultControls() {
return {
use_channel0: { // eslint-disable-line camelcase
default: "a"
},
color: {
default: {type: "color", default: "#fff700", title: "Color: "},
accepts: (type, instance) => type === "vec3",
},
...But since it does not hardcode any specific properties (missing required property map),
we can provide any values we want (including type change) as long as we pass the accepts check,
which in this case verifies the control outputs vec3 type.
Config values can be changed anytime. It is a good idea to not to force the renderer to copy the object,
this way you can share the configuration object active state all the time and modify it as needed.
For changes to take effect, you need to call viewer.drawer.rebuild(), same
for navigator if used. Moreover, use_* properties must call reset*() method. For filters, call resetFilters(...).
For change in mode or blending, call resetMode(). For changes in raster channel mapping, call resetChannels().
const shaderLayer = viewer.drawer.renderer.getShaderLayer('my-layer');
const config = shaderLayer.getConfig();
config.params.use_gamma = 1.0; // change gamma to 1.0
shaderLayer.resetFilters(config.params); // reset the use_gamma property to apply the change
viewer.drawer.rebuild();We might work on this more to simplify it.
When you override configuration to a custom shader set, you usually rely on tiled images to be present - but what some fails to load?!
You can use
VIEWER.addTiledImage({
tileSource: {type: "_blank", error: "Here goes your error detail."},
opacity: 0,
index: toOpenIndex,
});to render transparent placeholder data at the position of the missing tile source.
toOpenIndexis the index of failed image - advised is to open all images with explicit
index using addTiledImage to know it in advance. E.g., call this snipplet in error handler
of a parent addTiledImage call. You can access the error message later as viewer.world.getItemAt(toOpenIndex).source.error.
This drawer supports off-screen processing. You can either use the renderer directly, which is a bit harder,
or if you want to process current viewport in a different way, you can use $.makeStandaloneFlexDrawer(originalViewer)
and then call offscreeDrawer.draw(originalViewer.world._items). The new viewer can have different shader configuration,
rendering the same viewport in a desired manner. It's synchronized with the originalViewer data.
Once dev dependencies are installed, you can run the demo playground to see the renderer in action:
npm run devand open http://localhost:8000/test/demo/flex-renderer-playground.html in your browser.
- Bugfixing & getting ready for the first release
- Fixing tests: inherited from OpenSeadragon, they expect incompatible behavior
- Fixing coverage tests
- Adding support for WebGL 1.0 (fallback)
- Modularize ShaderLayers
- Implement modules (sample color, apply gaussian...) to connect together to create a ShaderLayer.
- Add support for concave clipping polygons.
- Adding support for better debugging & cropping
- For now, only convex polygons are supported
- Dynamic documentation that parses available shaders, controls, and shows what can be used where.
- Canvas2D proxy. People tend to use Canvas2D api to access the rendered data, which is currently not possible as the output canvas is native WebGL (or other rendering engine) element.
- Tainted Data. The purpose of this renderer is to draw advanced visualizations on GPU: if your data is not GPU-accessible, fix your data.
If you want to use OpenSeadragon in your own projects, you can find the latest stable build, API documentation, and example code at https://openseadragon.github.io/. If you want to modify OpenSeadragon and/or contribute to its development, read the contributing guide for instructions.
OpenSeadragon is released under the New BSD license. For details, see the LICENSE.txt file.
We are grateful for the (development or financial) contribution to the OpenSeadragon project.
