Combining Vector Graphics and Shaders with Solandra

Solandra 0.19 has APIs for rendering both fragment shaders (GLSL) and Solandra Sketches to offscreen Canvases in order to create images. These images can be combined within a Solandra sketch. Let's see how this works with a simple example.

How was this done? First let's render a fragment shader. Fragment shaders render per-pixel. So we don't describe shapes and lines, we instead must give a colour value for every single pixel. These values are calculated in a very low level way in the C-like GLSL. We have to figure out a 4d vector for each pixel where the first three values are the RGB values and the last is the alpha value. We can't even use values in our other code directly, we have to pass them in as uniforms.

Solandra takes care of setting up the context and wiring up the u_resolution uniform. To render something all you need to do is pass in a shader which contains a main function that writes to gl_FragColor. This creates a simple gradient where the red and green values are the relative x and y coordinates of the pixel.

const shaderImage = renderShader({
  shader: /* glsl */ `
        void main() { 
            vec2 r_pos = gl_FragCoord.xy / u_resolution; 
            gl_FragColor = vec4(r_pos.x, r_pos.y, 0.4, 1.0); 
        }
        `,
})

Now let's render something with Solandra offscreen. With Solandra we have access to much more powerful APIs and we don't typically care about individual pixels, we draw the entire scene. We will iterate over a set of PoissonDiskPoints, generating a hue from the index and drawing a circle.

const solImage = render({
  w: 1024,
  h: 1024,
  sketch: (s) => {
    s.forPoissonDiskPoints({ minDist: 0.05 }, (pt, i) => {
      s.setFillColor(i % 360, 80, 70, 0.7)
      s.fill(new Circle({ at: pt, r: 0.025 }))
    })
  },
})

Now let's combine these. We use withClipping to clip the shader to a circle and withBlendMode to overlay the Solandra image on top of the shader in an interesting way.

s.background(20, 40, 40)

s.withClipping(new Circle({ at: s.meta.center, r: 0.4 }), () => {
  s.drawImage({ image: shaderImage })
})

s.withBlendMode('overlay', () => {
  s.drawImage({ image: solImage })
})

Using Images from Shaders

We can also pass images into Solandra shader rendering. Let's first create a simple image of 'snow fall'. A black background and some white circles.

const solImage = render({
  w: 1024,
  h: 1024,
  sketch: (s) => {
    s.background(0, 0, 0)
    s.setFillColor(0, 0, 100)
    s.forPoissonDiskPoints({ minDist: 0.05 }, (pt) => {
      s.fill(new Circle({ at: pt, r: 0.01 }))
    })
  },
})

Okay how do we pass into a shader? That is very simple, the renderShader function takes an optional object of images.

images: {
  solImage
}

How do we use these images? With texture2D we can sample the image at a given coordinate. We can now do fancy per-pixel effects on the original images, so for example here we are taking the red, green and blue values from slightly offset locations to create an effect. The .r and so on just means take the first colour from this pixel i.e red.

const shaderImage = renderShader({
  shader: /* glsl */ `
        void main() { 
            vec2 r_pos = gl_FragCoord.xy / u_resolution; 

            vec2 r_pos_r = r_pos - vec2(0.01, 0.0);
            vec2 r_pos_g = r_pos - vec2(0.0, 0.01);
            vec2 r_pos_b = r_pos - vec2(0.01, 0.01);

            float r = texture2D(solImage, r_pos_r).r;
            float g = texture2D(solImage, r_pos_g).g;
            float b = texture2D(solImage, r_pos_b).b;

            gl_FragColor = vec4(r, g, b, 1.0);
        }
        `,
  images: { solImage },
})

// Actually render it:
s.drawImage({ image: shaderImage })