SourceForge.net Logo

Scientific Visualization Artist Tools

  1. News
  2. Overview
  3. Components
  4. Examples
  5. Download
  6. Installation
  7. Future Work
  8. Acknowledgements
  9. References


1. News

2008-04-08 Released v0.4.0 with the new tools vtkdump.py (textual dump of VTK files), pygribs.py (python generated rib scenes), png2spk.py (reverse engineer colorbars), and ncregrid3d (2D and 3D curvilinear-to-uniform grid interpolators).
Added new RenderMan shader RSZatmos.sl for an artistic, volumetric planet atmosphere (very pretty).
Small updates to xvtk.py (quick previewer of VTK files) to colorize texture coords, apply a texture, and scale spatial coordinates.
Added LightToRIB.py to convert a Blender light into a RIB light.
2007-10-09 Released v0.3.0 with the new tools mpir and seqck.py.
2006-09-01 Project hosted by SourceForge.


2. Overview

The SciVis Artist Tools (SVAT) is a collection of scripts, shaders and converters that will empower you to:

Yeah, but... aren't there already VTK-to-RenderMan and Blender-to-RenderMan converters? Absolutely, but could you show me published results that they meet all your needs? They didn't for me, and so here we are. For example, VTK ships a vtkRIBExporter class, but it poses the problem of exporting an entire scene in one go, in ASCII and without line support, and how often can you get away nowadays with visualizing complex scenes in a single render pass?

These tools are for you if:
These tools aren't for you if: Any ideas for a feature? Submit it here. Uncovered a bug? Submit a ticket here.


3. Components

Here's a quick break down of what each piece of the SVAT bundle does.

3.1 vtk2rib
3.2 Colormaps and Conversion
3.3 Shaders
3.4 Blender Tools
3.5 Miscellaneous Utilities

3.1 vtk2rib

It does what it's called, convert Visualization Toolkit (VTK) datasets into RenderMan Interface Bytestream (RIB). For input, it supports vtkPolyData, vtkUnstructuredGrid, and vtkPointSet with different flavors of RIB output, such as Texture coordinates, scalars and normals also go along for the ride where applicable.

3.1.1 vtk2rib Example 1

Here's what you can do in a Python script to export a vtkPolyData mesh with its scalars, texture coordinates and normals into a binary RIB file, just supply your own "CreatePolyData" function.
import vtk
import vtk2rib
vtk2rib.cvar.outputScalar = True
vtk2rib.cvar.outputNormals = True
vtk2rib.cvar.outputTCoords = True
vtk2rib.cvar.outputAscii = False
vtk2rib.cvar.outputFilename = 'ex1.rib'
vtk2rib.vtkPolyDataToRIB(CreatePolyData().__this__)

3.1.2 vtk2rib Example 2

Here's a variant that exports a point cloud as spheres using the same radius (although they can be scaled by scalars by using vtk2rib.cvar.outputScalarAsSphereRadius).xs
import vtk
import vtk2rib
vtk2rib.cvar.outputSphereRadiusScale = .1
vtk2rib.vtkPointSetToRIBSpheres(CreatePointSetData().__this__)

3.2 Colormaps and Conversion

Other than geometry, the second most important aspect to Sci-Vis is probably mapping colors to your data, and we have many ways to do that to ultimately render them with RenderMan. First you'll need to define a colormap and then create a RenderMan shader that can use it. There are three such ways.

3.2.1 Uniform Colormap

This is the simplest approach which assigns a series of colors across an evenly spaced scalar range. This minimizes the number of changes needed to get something working, but may not work well when your data histogram is much more clustered or logarithmic. You can generate a simple shader with this command:
m4 -DNUMCOLORS=6 RSZUniformLUTRGBA.sl > uniformlut6.sl
And then specify the scalar limits and colors in your scene's RIB file:
Surface "uniformlut6"
"float smin" [0.0]
"float smax" [1.0]
"uniform color rgb[6]" [ 0 0 0  1. 0. 0.  .7 1. 0.  0. 1. 0.7  0. 0. 1.  0 0 0]
"uniform color a[6]" [ 0 0 0  0. 0. 0.  .3 .3 .3  .6 .6 .6  1 1 1  0 0 0]

3.2.2 vtkLookupTable

You can also choose to use a linear colormap defined by a vtkLookupTable. So in a Python script, use these commands to generate a snippet of RIB that resembles what you saw in section 3.2.1, excluding the shader's name because we implemented many shaders that support colormaps.
from vtkLookupTableToRIB import vtkLookupTableToRIB
# Define your lut instance...
# Now print the RIB code.
print vtkLookupTableToRIB(lut, n=6)

3.2.3 ASCII Colormap

Converting the color, opacity, and relative scalar range from a simple text file can be quite powerful because the colormapping doesn't need to be uniform. Here's how to use a script to generate a 256 color table
spk2rib.py ../colormaps/jet.spk -1 1 256
The color entries are simply described in ASCII by jet.spk with 5 columns (scalar percent, red, green, blue, opacity) so it's easy to make customizations:
0       0       0       50      100
12.5    0       0       100     100
37.5    0       100     100     100
62.5    100     100     0       100
87.5    100     0       0       100
100     50      0       0       100
Notice that the output doesn't have any endpoint spline padding as was the case for the previous 2 uniform colormap approaches. Put this output into your scene's RIB where you defined your shader that understands colormaps. You can create such a shader as described in the shader section.

3.2.4 Reverse Engineer Rasterized Colorbar to ASCII

If you come across a nice scientific visualization with a clever color mapping, why not re-use it? After cropping out the colorbar strip into a PNG image, you can apply this script to generate an ASCII file in the SPK format like shown above in Sections 3.2.x. However, it's your job to deal with any non-linear scalar mapping. This will only extract the color components.
./png2spk.py colorstrip.png > pirated.spk

3.3 Shaders

Here's a growing number of RenderMan shaders intended to let you break into REYES rendering with your Sci-Vis problems.

3.3.1 A Generic Colormap Material

You'll need to generate this material and include it in any of your colormap-aware shaders. The reason for this is because the RenderMan Shading Language (RSL) doesn't support dynamic arrays, so before compiling, you'll need statically allocated arrays for your color specification, and that depends on how many color entries you defined. Here's how to create the include file when using 6 colors:
m4 -DNUMCOLORS=6 RSZMaterialLUTRGBA.sl > RSZMaterialLUTRGBA6.sl

3.3.2 Colormapped Shaders

After creating the basic colormap material, you can create a shader using a plastic reflectance model:
m4 -DNUMCOLORS=6 RSZPlasticLUTRGBA.sl > lutplastic6.sl
The same can be done for similar colormap-aware shaders. These include:

3.3.3 Volumetric Shader

3.4 Blender Tools

Blender can be invaluable to create camera flyovers or to model and keyframe completely new 3D objects, which can help to embellish your Sci-Vis scenes. At the very least, Blender can be used as a previsualization tool when coupled with the VTKBlender module.

3.5 Miscellaneous Utilities

Some programs crop up to fulfill exceptional needs outside the scope of the other tools. These can be in C/C++ for performance reasons, or to link with a particular API, or in a scripting language to just get the job done. Don't worry, there's a Makefile to compile where applicable.


4. Examples

Aside from the unit tests that ship with SVAT, you can see firsthand how some scenes were created. Observe how the input files are related and what its like to render it yourself.

4.1 Example 1: Displaced Surface

This scene is so simple, it can be thought of as a "Hello World" of displacement mapping. We just apply an image texture for colors and a grayscale image to displace the square plane. The Blender file allows us to graphically place the camera and export it.
After downloading the demo1.tar.gz , run these commands to render it with 3Delight. Be sure that the renderer can find the RSZsupertexmap shader file.
tdlmake texture.jpg texture-mipmap.tif
tdlmake displacemap.tif displacemap-mipmap.tif
renderdl -d demo1.rib

4.2 Example 2: Colormapped Data

Here we'll build upon a displaced spherical surface with colormapped data from a simulation with a small embellishment of coastlines. Animations of this graphic may be found at GFDL's Visualization Gallery.
After downloading demo2.tar.gz and demo2-coastline.tar.gz , run these commands to render it with 3Delight:
# Create shader with SVAT distribution.
m4 -DNUMCOLORS=5 RSZMaterialLUTRGBA.sl > RSZMaterialLUTRGBA5.sl
m4 -DNUMCOLORS=5 RSZPlasticLUTRGBA.sl > RSZPlasticLUTRGBA5.sl
shaderdl --dso -O3 RSZPlasticLUTRGBA5.sl

tdlmake displacemap.tif displacemap-mipmap.tif
renderdl -d demo2.rib

4.3 Example 3: Colormapped Data with Particle Texturing

We'll combine colormapped data with textured particles to create a cloud effect, plus our background will use a procedural tile pattern and 3D objects modeled in Blender. Animations of this graphic may be found at GFDL's IPCC Models and Hurricanes research webpage.
After downloading demo3.tar.gz , run these commands to render it with 3Delight:
# Create shaders with SVAT distribution.
m4 -DNUMCOLORS=6 RSZMaterialLUTRGBA.sl > RSZMaterialLUTRGBA6.sl
m4 -DNUMCOLORS=6 RSZgcloudLUTRGBAfBm.sl > RSZgcloudLUTRGBAfBm6.sl
shaderdl -I/usr/local/renderman/arman/shaders --dso -O3 RSZgcloudLUTRGBAfBm6.sl

m4 -DNUMCOLORS=8 RSZMaterialLUTRGBA.sl > RSZMaterialLUTRGBA8.sl
m4 -DNUMCOLORS=8 RSZPlasticLUTRGBA.sl > RSZPlasticLUTRGBA8.sl
shaderdl --dso -O3 RSZPlasticLUTRGBA8.sl

shaderdl -I/usr/local/renderman/arman/shaders --dso -O3 RSZTiledPlastic.sl

# The arrow mask was used to make animated arrows fade into the scene.
tdlmake arrowmask.tif arrowmask-mipmap.tif
tdlmake land.tif land-mipmap.tif
tdlmake landdisp.tif landdisp-mipmap.tif

# Render the deep shadow map. The input RIB is the same as demo3.rib, but with some lines commented so only the particles create shadows.
renderdl -progress demo3dsm.rib
# Now render the scene's background elements with the shadow map included.
renderdl -d demo3.rib
# Now comment out the grid, land and data objects and render the clouds and arrows.
renderdl -d demo3.rib
# Now you can composite the results.
composite demo3fore.tif demo3back.tif demo3.tif

4.4 Example 4: Batch Rendering on an MPI System

The first example renders 2,000 frames with Blender on a 512 CPU computer. Notice how mpir uses a command-line template. You specify a frame range length with the '-g' option. Progress is echoed as a table.
# Start the multi-cpu rendering.
mpirun -np $NSLOTS mpir -S 15 -s 1 -e 2000 -g 15 'blender -b scene.blend -o /tmp/ -F PNG -x 1 -t 1 -s %start -e %end -a'

# Status is echoed every 15 seconds using the '-S' option.
-----------------------------------------------
Task |  PE  | Start | End   | Time  | Time Avg
000  | 0001 | 0001  | 0015  |       |
001  | 0002 | 0016  | 0030  |       |
...
------------------------------------------------------
Elapsed  | Computed |   Min    |   Avg    |   Max    | Progress
00:02:02 | 00:00:00 | 00:00:00 | 00:00:00 | 00:00:00 | 0/2000 (0%)
This will also render frames, but only those specified in a text file which can be generated manually or by seqck.py.
echo 1 15 22 25 > frames.txt
mpirun -np $NSLOTS mpir -S 15 -i frames.txt 'blender -b scene.blend -o /tmp/ -F PNG -x 1 -t 1 -s %start -e %end -a'
Here we use mpir to do image cropping and rolling with ImageMagick. For string interpolation of %start, we specify the %d format using the familiar C-printf syntax.
mpirun -np $NSLOTS mpir -S 30 -s 1 -e 2000 -g 1 -f '%d' 'f=`printf %04d.png %start`; convert \( /tmp/$f -crop 3999x1998+1+1 \) -roll +2000+0 +repage /tmp2/$f'

4.5 Example 5: Checking for Missing Animation Frame Files

In the event that your batch rendering didn't complete, you can check your list of files produced to find what's missing. seqck.py will scan your filenames to see if any name is missing from a sequence. For example, if frames 1850 and 1900 are missing from a rendering of 2,000 frames, here's what we'd get:
seqck.py -s 1 -e 2000 -i '/tmp/%04d.png' -o '%04d'
1850
1900


5. Download

Project Tree: svat-0.4.0.tar.gz


6. Installation

  1. There are prerequisites, but some are optional.
  2. Download and untar the project tree.
    tar xvzf svat-*.tar.gz
    
  3. Change into the top SVAT directory and modify the file Makefile.inc. You'll have set the variables ROOT and PREFIX. If you choose to use 3Delight as your RenderMan renderer, also set DELIGHT.

    Change into the src/ directory, edit the Makefile and set the paths to where your VTK libraries and include files reside. Change into the ../vtk2rib/src/ directory, edit the Makefile and set the paths to where your VTK libraries and include files reside. I need to consolidate this into Makefile.inc.

    Finally, you can compile these branches.
    cd vtk2rib/src
    make
    make check
    make install
    cd ../../src
    make
    make check
    make install
    cd ../shaders
    make 
    make install
    
    Now you'll need to setup your environment so that the installation paths are 1) visible for shader compiling 2) python modules are importable and 3) binaries are found in the execution path. Assuming you installed into /usr/local/svat you should add these lines depending on your shell:
    # For BASH (~/.bashrc).
    export SVATHOME=/usr/local/svat
    export LD_LIBRARY_PATH=/usr/local/svat/lib:$LD_LIBRARY_PATH
    export PYTHONPATH=/usr/local/svat/lib:$PYTHONPATH
    export PATH=/usr/local/svat/bin:$PATH
    
    # For CSH (~/.cshrc).
    setenv SVATHOME /usr/local/svat
    setenv LD_LIBRARY_PATH "/usr/local/svat/lib:${LD_LIBRARY_PATH}"
    setenv PYTHONPATH "/usr/local/svat/lib:${PYTHONPATH}
    setenv PATH "/usr/local/svat/bin:${PATH}"
    


7. Future Work


8. Acknowledgements


9. References

  1. RenderMan Academy - Some tutorials to get started with RenderMan.
  2. RenderMan Repository - A wealth of tutorials and shaders for RenderMan. You can also find the "Advanced RenderMan" book's source code here.
  3. rib-mode - For (X)Emacs, a syntax highliting, auto-indentation major mode for RIB files for the (X)Emacs text editor written by yours truly.
  4. Scientific Visualization using Tools and Techniques from the Motion Picture Industry - A presentation describing SVAT given by yours truly at the 2007 Raytheon Software Symposium. Available in PowerPoint or PDF (15 MB). There are animations on slide 10 and 11, and probably only play on Windows.


Copyright © 2007-2008 Remik Ziemlinski
RenderMan® is a registered trademark of Pixar.