Shader experimentation: Cut-out billboard particles with depth

This is something that has been bothering me for a while. The majority of billboard particles in games have completely flat intersections with floors and walls. A number of solutions have been designed to deal with this; soft particles, or particle collisions to keep them from intersecting walls or floors. It's been on my mind for a while, what if there was a way to have billboard particles write custom values to the z-buffer, so that they can appear to be 3D even when intersecting other particles and objects? Now obviously this isn't going to work with particles intersecting each other if the particles are transparent (as they do not write to the z-buffer), so to start with this experiment (after determining that there is a way to modify z-buffer values on a per-pixel level, and yes, there is) I decided to make a clone of Unity's legacy "soft edge unlit" cutout particle shader (I chose unlit because, if you're experimenting, it's best to start simple).

After quite a bit of experimentation and research, I succeeded. The hardest part was figuring out the formula for adding a specific distance to the z-buffer in eye-space (eye-space being world space proportions, but with the z-axis pointing in the direction of the forward vector of the current camera, and the x and y axes to the left and up vectors, respectively). The z-buffer varies from between 0 (the near clip plane of the camera) to 1 (the far clip plane) and these values are stored non-linearly in respect to their eye-space value! Unity gives you a macro to convert from z-buffer values to eye space Z values (LinearEyeDepth()) but it doesn't have a function for doing the reverse. I was able to tract down the LinearEyeDepth macro in the shader source code and invert the formula, allowing me to convert from z-buffer space to eye space, subtract an amount from the z value (determined by the depth texture attached to the material) and convert it back to z-buffer space.

There were some problems at first. With only 8 bits of precision on the height map, the intersections between the spheres looked really ugly:

I through together a quick script to generate a 32 bit spherical heightmap:

Upon being attached to a gameobject in the scene, and pressing play, this deposited the following image in the project folder:

I spent a while trying to get the shader to read in the texture as an unsigned 32 bit integer and do some bitwise math to turn it into the proper value. After succeeding and getting a result that didn't look any better than it did previously, I took a break, and coming back to it I discovered the ability to change the image format in the import settings to a 16 bit single channel. Miraculously, it took the mess you see above and simply made it into a simple one-channel height map with much more precision than the one I was using previously.

I put that texture into the original shader, reading it in like normal (with none of the fancy bitwise math I was trying to do earlier), and the problem was greatly reduced!

On the left are particles using my shader, and on the right are particles using the cut-out shader I derived my shader from.

You can barely tell that what's on the left are billboard particles, a bunch of flat planes being rendered to face the camera - they look like a bunch of high poly spheres!

And of course, since it uses a height map, you could do this with any shape you wanted, not just spheres (although spheres will naturally look the best, as they maintain their shape when you rotate the camera).