Open Shader Language (OSL) is a C++ based open source shader language developed by Sony Picture Imageworks. The most immediate way to work with OSL is to simply take an OSL shader someone else wrote and use it. Part I of this tutorial therefore will offer a number of OSL textures and materials which can be used to extend the functionality of Vray beyond what it can normally do.

Note that OSL in Vray does have some limitations. For example it does not currently work with IPR/RT. Additionally, it will tend to crash when multiple renderer plugins are loaded into a scene (for example Vray + Renderman + Mental Ray). This crash typially occurs either while rendering or when the Hypershade is opened (which renders the shader balls).

Part I: OSL Examples

Jump to shader section:
Iridescence (texture)
Metal (texture)
Fabric (material)
     Fabric recipe for a VrayMtl
Skin (material)
Matcap (texture)
Hair Color (texture)

    Iridescence (texture)

      This texture reproduces the effect of thin film interference on a surface. This can be used to reproduce the colored iridescent reflections on soap bubbles, or a host of other things: pigeon feathers, insects, and so on. It's based on the paper Thin Film Interference for Computer Graphics.

      To use it, download (right-click "save link as") iridescenceTex.osl and load it into a VRayOSLTex, then connect that to the reflection color slot of a VRayMtl with Fresnel turned off (the shader does its own Fresnel calculations).

    Artist Friendly Metal reflections (texture)

      The Fresnel on a VrayMtl is a simplified Fresnel equation which works for dielectric materials such as plastic or glass, but not for metals (conductors) which require a more complex Fresnel calculation involving both the index of refraction (n) and extinction coefficient (k). This is the reason that the values for metallic materials listed on a site like http://refractiveindex.info seem wrong (for example it lists aluminum as IOR: 0.97). The physics are right, but only if you have both the n & k parameters in your Fresnel calculation and the VrayMtl only has n so the reflective IOR for metals are wrong. Further, as the reflective IOR is raised above 2 using this simplified equation, it begins to get darker on the edges rather than brighter the higher you go.

      There's two ways to address this: First is the super hard technical way, and second is a more intuitive artist friendly way.

      Hard: Calculating reflective IOR for metals from measured data

        Chaos has a Complex Fresnel shader which can be used to input the measured data for the n & k parameters of complex Fresnel.

        Here's the process. The visible spectrum of light is measured in wavelengths. The human eye can see wavelengths from about 390 to 700 nanometers (nm). Because metals such as gold have colored reflections, we need to input the wavelength values for red, green, and blue to get the corresponding IOR values. Using this Wavelength to RGB calculator we can see that red is 645, green is 510, and blue is 440. Next we go to http://refractiveindex.info, choose the measured data we want (for example gold) and enter the wavelength we got for red, green and blue. These are in nanometers (nm), but refractiveindex.info wants micrometers (µm) which would be red 0.645, green is 0.51, and blue is 0.44. From this we will get the n and k values for the red, green, and blue wavelengths which we can then input into our complex_ior shader. For example, here are the values for gold:

        n: 0.18627, 0.75314, 1.4324
        k: 3.3758, 1.9238, 1.7915

      Easy: An artist friendly approach to complex Fresnel for metals

        While complex Fresnel using measured data is more accurate for metals, it can be unintuitive for artists to work with since you must enter numbers, rather than working with colors. If you want exactly the numbers you type in from the measured data, you're fine, but if you want to tweak the colors a bit (which of course you will!), you're in trouble.

        That's where the artist friendly shader comes in. This shader remaps the complex Fresnel equation for n (refractive index) and k (extinction coefficient) to the more intuitive reflectivity (r) and edge tint (g) which are input as RGB colors. In other words, you enter in a color for the metal and another for the edge tint, and the shader uses the complex Fresnel equation of n and k to give you proper Fresnel reflections for metal.

        The shader is based on the paper Artist Friendly Metallic Fresnel written by Ole Gulbrandsen from Framestore. To use it, download (right-click "save link as") AFMFresnelTex.osl and load it into a VRayOSLTex, then connect that to the reflection color slot of a VRayMtl with Fresnel turned off (the shader does its own Fresnel calculations).

    Fabric (material)

      This is an artist friendly fabric shader. To use it, download (right-click "save link as") DFfabricMtl.osl and load it into a VRayOSLMtl.

      Shader description: The approach of the shader is phenomenological, based on observances of the material properties of fabrics and how they react to light. The shader is an anisotropic Blinn BRDF with added sheen and tint terms to capture how the tiny fibers of fabric pick up light at incident angles. Additionally the tint affects the specular to approximate how the specular color of fabrics are often tinted by the diffuse color. The sheen color can also be used to darken the sheen as some fabrics (for example black nylon) are darker at the edges, rather than brighter. The sheen fall-off has six options to choose from in order to get the desires sheen of the particular fabric: linear, exponential, smooth (hermite), sigmoid (s-curve), square root, and Fresnel.


        Sheen parameters:

      • Sheen Color: Determines color of sheen. Defaults to white. Sheen color can be black for nylon sheen, but be aware that when set to pure black (0,0,0) as with all color swatches, the default color will be used. Setting to almost black (0.01,0.01,0.01) will work fine.
      • Sheen Opacity: Determines amount of sheen effect as fibres pick up light at incident angle. Can be used to disable the sheen.
      • Sheen Tint: Tints both the sheen and specular color with the diffuse color.
      • Sheen Ramp Type: Six types of interpolation for sheen: Linear(1), Exponential(2), Smooth(3), Sigmoid(4), Square Root(5), Fresnel(6).
        Diffuse parameters:

      • Diffuse Color: Same as in a VrayMtl. Defaults to deep red when swatch is set to black.
      • Diffuse Weight: Same as in a VrayMtl. Can be used to disable the diffuse.
      • Diffuse Roughness: Gives the diffuse a soft powdery look, typical of cloth.
        Spec parameters:

      • Spec Color: Same as in a VrayMtl. Defaults to white when swatch is set to black.
      • Spec Amount: Same as in a VrayMtl. Can be used to disable the spec.
      • Spec Glossiness: Same as in a VrayMtl.
      • Spec Tail: GTR tail falloff set to 1.0 or below gives nice cloth-like sheen to the spec.
      • IOR: Determines the strength of Fresnel reflections; fabric generally has low frontal reflections.
        Adv Reflection parameters:

      • Subdivs: Same as in a VrayMtl. This option is disregarded when "use local subdivs" is off in the render settings.
      • Trace Reflections: Same as in a VrayMtl. Disabling this will result in spec only without reflections
        Anisotropy parameters:

      • Anisotropy: Fabric is anisotropic due to weave.
      • Aniso Rotation: Same as in a VrayMtl. A value of 0 = 0 degrees, 0.125 = 45 degrees, 0.25 = 90 degrees, 0.375 = 135 degrees, and 0.5 = 180 degrees. So the range is 0-0.5 which gives 0-180 degree rotations.
      • Texture Gamma: Gamma correction for diffuse color.
      GUI limitations: Note that with the current implementation of OSL in Vray colors that can have texture map inputs default to black. They are set to use grey for diffuse and white for spec by default, but the swatch in Maya will still look black. Feel free to change the color to whatever you like. Setting it to black will use the default values (grey for diffuse and white for spec). Further, since OSL widgets are not implemented yet, the dropdown menu for ramp_type does not work, and you need to manually choose one of the 6 options described above.

      A bug with OSL and vrayBumpMtl: There is currently a bug in Vray, when an OSLmtl is plugged into a VrayBumpMtl, resulting in the shadows on the bump being hard and aliased (one could say posterized), rather than soft and feathered. This bug affects both bump and normal maps. Because of this, bump mapping has been disabled on the shader. As a workaround simply plug the OSLMtl into a VrayBumpMtl.

      Recipe for cloth with a VrayMtl

      While not as interactive, it is possible to create a reasonable cloth material using a VrayMtl and a few tricks. The first step is to set the material attribute as follows:

      • Diffuse Color:
        1. Create a RemapHSV node and connect your diffuse texture to it. Increase the value and lower the saturation.
        2. Create a Vray Falloff map, and plug the diffuse texture into its front color, and your RemapHSV into its side color.
        3. Set the Falloff type to "perpendicular/parallel" and change the interpolation type of it's mix curve to "smooth" for both points.
        4. Finally, plug the Falloff map into the diffuse color of your VrayMtl.
        (This will create a sheen effect mimicking cloth fibres picking up light at incident angles. Sheen is visually by far the most import aspect of our cloth shader recipe)
      • Diffuse Weight: 0.8
      • Diffuse Roughness: 0.5 (Gives the diffuse a soft powdery look, typical of cloth)
      • BRDF Type: Blinn
      • Reflection Color:
        Plug the RemapHSV node you made above into the reflection color.
        (This will produce spec tinted by the diffuse color, mimicking the general appearance of cloth specular)
      • Reflection Amount: 0.5
      • Reflection Glossiness: 0.6
      • Reflection IOR: 1.33 (Produces low frontal reflections typical of most fabrics)
      • Anisotropy: 0.25 (fabric is anisotropic due to the weave)

      You will of course need to have good diffuse and bump maps to make the cloth convincing. Note that with high frequency bump maps, typical of a cloth weave, Vray can get an error where the diffuse terminates in a sharp line rather than gradually over the surface. This is apparently caused by an error in the viewport subdivisions. Apply Vray subdivisions to the geo's shape node instead (add attribute > subdivision), and it should work fine.

    Wavelength Dependent Subsurface Scattering (material)

      Fig 1

      The following OSL material emulates the skin shader made by Tony Reynolds, which employs layered subsurface scattering modulated by red, green, and blue wavelengths. Additionally the shader implements a two-lobe spec to capture the subtle variation of spec on human skin. To use it, download (right-click "save link as") DFskinMtl.osl and load it into a VRayOSLMtl.

      Shader description (and a little bit of physics)

      Each wavelength of light penetrates the skin to a different depth. If we simplify our visible spectrum to three colors (red, green and blue) the mean free path (the distance a photon travels through a particular material before it gets scattered or absorbed) will take a different value for each one of these colors. For human skin red goes the deepest, then green, and blue is quite shallow.

      Based on this, the shader separates the main color texture (the Subsurface Color) into its red, green and blue components, and pipes these into three subsurface shaders, which are then combined together. The depth of each these three subsurface shaders (i.e the mean free path) is then driven by the corresponding red, green, and blue channel of the "RGB Scatter Depth" color. To speed things up the green and blue are combined, so the shader combines two sss closures together (red + green&blue) rather than three. The idea here is that the dominant scatter color for all humans is red (since we all have red blood under our various skin colors).

      In other words, the RGB Scatter Depth is not actually a color per se, but rather represents the scattering coefficients in the volume for the red, green and blue wavelengths respectivly. So setting the red channel of the Wavelength RGB Depth color to 1.0 means the depth of the red wavelength is at 100% while setting it to 0.7 means it is 70%, and so on for all three RGB channels. Note in the image below that the red channel is softer than the green or red since it has more depth.

      Fig 2. RGB scatter depth values: 1.0, 0.4, 0.2

      One added benefit of this approach is that it virtually eliminates the "green error" that can appear on thin parts of an SSS2 material when the scattering amount it set to large values since it linearly blends multiple subsurface shaders together, rather than the more complex interaction of the subsurface and scatter colors in the SSS2 material, which, while more physically correct, can be unintuitive for artists to work with.

      Overall, the focus of this shader is to prioritize artistic control over physical correctness. One drawback of layered skin shaders like the VraySkinMtl is that a user needs to mix different, often unintuitive, colors together to arrive at the desired skin color. As autodesk says of their layered Misss* shader, "Each layer had it's distinct color, which made it difficult to aim for a given "final" color, and changing the balance between the layers threw the color off." Similarly, a drawback of the SSS2 material is that because of the complex math involved in subsurface scattering calculations an artist may select a particular color and will get a completely different one in the render.

      In contrast, with this shader the user simply inputs a single texture map of the skin into the Subsurface Color and that's what you get. Likewise, the user sets the color for the Wavelength RGB Depth that they want to see in the bleed (keeping in mind that this also affects the scatter depth) and you get that too. WYSIWYG.

      Single scatter/Diffuse

      The shader contains a diffuse parameter set to grey in order to simulate single scattering in realistic skin. In other words, this is like a very rough specular reflection, and so should be colorless since skin is dielectric. Because of how OSL works with linear blending, this works better than the diffuse in a SSS2 which effectively cancels the sss. in contrast, here a value around 0.5 works nicely. For more cartoony looks you can set the amount to 0 to turn it off if desired.

      Two-lobe Specular

      The shader combines two specular lobes together. A "sharp" specular lobe, using a GGX microfacet BRDF (a GGX is equivalent to a Generalized Trowbridge-Reitz (GTR) BRDF with its tail falloff parameter locked at 2.0), and a "soft" specular lobe using a Blinn BRDF. The OSL closure balances these two spec lobes with the SSS so the total does not exceed 1.0, making it energy conserving (i.e. the total amount of reflected light cannot be more than the incoming light). If you want to get a little technical, the renderer sums the weights of the closure components, and if they exceed 1.0, it divides the weights by their sum, keeping them in the same balance but ensuring that they never sum to more than 1.0.

      The spec glossiness can be driven by connecting a black and white map (see Fig 3.1 below) to the Gloss Mask parameter. The white parts of this map will get the "max glossiness" value, the black parts will receive the "min glossiness" value, and the grey areas will get a value inbetween the min and max values (see Fig 3.2 below). If no map is provided the shader will default to use the max glossiness value. This allows the artist to define parts of the face that are more glossy, such as the lips and the T-zone. Note for instance in the image below (Fig 3.3) how the lips and nose appear smooth and shinny since the mask is white here and receiving the max gloss value, while the area above the lips is rough since the mask is black here and thus receiving the min gloss value.

      Fig 3. A Black and white gloss texture map and the resulting specular variation from the Min/Max Glossiness values.

      The two-lobe spec setup further adds detail, combining a sharp spec (controlled by the min/max glossiness values) together with a softer spec controlled by the "spec bloom" amount. The higher the bloom amount, the larger the spec becomes, so it is akin to a "roughness" parameter. The bloom is also affected by the gloss mask. Both spec lobes can be isolated (i.e. turned off) through the use of their "amount" sliders.

      Fig 4. OSL sharp and bloom spec lobes. Model courtesy of the Wikihuman Project


      The shader does not have bump, due to a current limitation in OSL for Vray (or to state things differently, the Vray bump is quite sophisticated, and so implementing it in OSL is not trivial). To get bump, just connect the OSLMtl to a VrayBumpMtl.

      As an additional tip to users, having a nice bump map--including the micro detail between the skin pores-- is essential to effectively capturing human skin. Note in the image below (fig 5) how you can see details in the spec that is much smaller than even skin pores. The two spec lobes are essential to capturing this detail. One way to create these maps is to sculpt the detail in a modeling package and extract a floating-point EXR displacement map. However, it is often easier to simply paint these as bump maps in a program such as Mari, using different layers for the pores and the micro details inbetween the pores. The trick to getting this kind of detail in the bump (as opposed to displacement) is to lower the bump delta scale (say from 1 to 0.1) which produces a sharper result comparable to the fine detail look of a displacement map. Another approach is to separate the micro detail and apply this to a spec map.

      Fig 5. Micro detail on skin from photometric scans. Images from Lee Perry-Smith's Infinite Realities


           Fig 6. Attribute Editor

        General parameters:

      • Transparency: Controls the transparency
      • Depth Scale: Global control for the depth of the subsurface scattering
      • Overall Brightness: Controls the brightness of the colors. Can be used to disable the sss and diffuse.
      • Diffuse Color: Simulates single scatter with a colorless Labertian diffuse.
      • Diffuse Amount: Controls the diffuse amount. Can be used to disable the diffuse.
        Subsurface parameters:

      • Subsurface Color: The main color of the material. This is where your color texture goes. Default RGB values: 0.85,0.75,0.65
      • Subsurface amount: Controls the amount of shallow scatter, as opposed to deep scatter. Can be used to disable the sss.
      • RGB Scatter Depth: Controls the percentage of scattering for the red, green, and blue wavelengths, which also affects the scatter color. Default RGB values: 0.8,0.2,0.1
      • Texture Gamma: Gamma corrects the subsurface color
        Reflection/Spec parameters:

      • Spec Color: The color of the specular highlights (skin is dielectric, so this should be white)
      • Gloss Mask: Black and white map to modulate between the min/max glossiness values below.
      • Spec Total Amount: controls the total amount of both spec lobes. Can be used to disable the spec.
      • Sharp Amount: Amount of the sharp spec. Can be used to disable the sharp spec.
      • Bloom Amount: Amount of the soft spec. Can be used to disable the bloom spec.
      • Gloss Min: Glossiness value for the black parts of the Gloss Mask map
      • Gloss Max: Glossiness value for the white parts of the Gloss Mask map
      • Spec Bloom: Controls the size of the spec bloom. Larger values increase the size of the bloom/tail.
        Advanced Reflection parameters:

      • Reflection IOR: reflective index of refraction for the spec
      • Reflection subdivs: Subdivs for reflection (SSS in OSL for Vray does not currently have subdivs). This option is disregarded when "use local subdivs" is off in the render settings.
      • Trace Sharp Reflections: Enables raytraced reflections for the sharp spec BRDF (disabling this produces the spec component only)
      • Trace Bloom Reflections: Enables raytraced reflections for the bloom spec BRDF (disabling this produces the spec component only)
      • Use Single SSS: Disables the RGB Wavelength layered SSS, and uses a single subsurface term instead (equivalent to the VraySSS2Mtl). Not as nice, but faster.

    Matcap (texture)

      Matcap stands for "material capture" and is a method of real time shading made popular by Zbrush. All the shading information (including shadows and lights) are baked into the texture in the form of a "lit sphere" which is applied to the geometry based on the camera normals.

      The baked lighting of a matcap works well so long as the camera does not move, since when it does it is revealed that the lights and shadows are moving with the camera. Because of this, matcaps are best suited for "toon shading" like a Bugs Bunny cartoon where light direction is not taken into account.

      To use it, download (right-click "save link as") MatcapTex.osl and load it into a VRayOSLTex, then connect that to a VRayLightMtl. You will also need a litsphere texture map to connect to the matcap OSL texture. You can download this zipped archive with several example LitSphere textures or make your own.

      Below you can see several example of lit sphere textures and the resulting matcap render. Lit sphere textures work best that are smooth, rather than ones with lots of fine detail. For example, note that the "drawn" matcap below has some stretching of the pencil lines, whereas the smooth design "graphic" matcap reads perfectly.

    Hair Color (texture)

      This texture reproduces physically accurate hair color based on melanin content, based on the paper "An Energy-Conserving Hair Reflectance Model". The hair color is determined by the "melanin amount" color. Note that while melanin is a color input, the value should be black and white as the shader uses the luminance value (osl in Vray currently does not have sliders for floats so we use a color as workaround). A value of 0.0 melanin content results in white hair, 0.2 is blond, 0.3 is auburn, until you get to 1.0 for black hair. The "Dye Color" can be used to emulate dyed hair, acting as a multiplier. Note that the dye color by default appears black, but is actually set to white (no dye).

      To use it, download (right-click "save link as") DFhairTex.osl and load it into a VRayOSLTex. The resulting texture hair color goes in "transmission color" and "diffuse color" of a VrayHairMtl3, with the "diffuse value" ideally less than 0.5. The secondary specular is the hair color multiplied with itself (which is what the "lock secondary" checkbox in the VrayHairMtl does).

    Part II. Writing OSL Shaders
All content © copyright Light Collab.