Pretty much everyone I know has asked me the same question: “Why are you doing your PhD now?”

It’s a fair question, so let me rewind a bit.

Starting in graphics

I first got interested in ray tracing in 2013, after taking a basic graphics course. I asked my professor where I could learn more about how movies are made, and he pointed me towards PBRT. Unfortunately, I struggled a lot—I had almost no math background and zero knowledge of C++. But I stuck with it. I did my final year project in offline rendering, started reading papers, pursued a master’s degree (also rendering-related), and thought I’d continue further. Sadly, at the time there weren’t any opportunities in Malta. Things changed in 2015 when I went to SIGGRAPH as a student volunteer. By a lucky chain of events, in August 2017 I joined MPC Film in London. I met a lot of wonderful people and engineers, but there wasn’t much of a rendering research community there, apart from the shader developers. I did get pulled into a rendering project, which was a great experience, but I was often the one most excited about rendering papers and research, without many peers to share that with.

By the summer of 2019, I was interviewing at Wētā while also speaking with the late Jaroslav Křivánek about possibly starting a PhD with his group in Prague, focusing on path guiding. The first thing he asked me was:

“You’re already working in the industry—why do you want to do a PhD? You’ve already got the job in rendering!”

It was a valid point. But what I was realizing was how different production and research really are. In production, you rarely get the time to dive deep into a problem—priorities shift constantly depending on the needs of artists. I wanted the space to properly educate myself in light transport and to dig into problems deeply enough to offer real solutions.

Joining Wētā

When I got the Wētā offer as a Researcher, Jaroslav actually encouraged me to take it—he said there wouldn’t be a better education than being in the thick of things at Wētā’s rendering department. So I moved to New Zealand. Two months later, the pandemic hit, borders closed, and my partner couldn’t join me. I spent two years waiting for the borders to open. In the meantime, I’d met some incredible people in Wellington and learnt heaps from all the people at Wētā. It truly is a brilliant place to work at. But, as I waited for the borders I realized I was getting the yearning for doing a PhD again and it didn’t look like my partner was going to be able to come to New Zealand. So, I decided to move back to Malta, and after discussions with colleagues and department heads, the plan became: I’d work remotely for Wētā for a while, then start a PhD in September 2022, focusing on a long-standing research problem we’d encountered in production. The pull towards a PhD came from the same reasons as before. At Wētā, I noticed how people who had done research early in their careers as grad students seemed to operate on a different wavelength. To be trusted with a research project in industry, you usually need to have gone through that training yourself, which I hadn’t.

PhD Life

It’s now been three years since I started my PhD. The first year was rocky, filled with classes and lots of experimentation. In the second year, I hit the ground running with our Dwivedi project and submitted it to SIGGRAPH Asia. We got rejected, but the feedback was valuable. I spent the next nine months refining it before resubmitting to EGSR, where it was accepted at Computer Graphics Forum. Here’s me presenting the paper this year in Copenhagen:

So why am I doing a PhD?

Because I love rendering and want to learn it as deeply as possible. I don’t just want to implement other people’s work - I want to do novel research and explore ideas no one has tried before. I noticed that researchers have a different way of thinking about a problem - and there is a certain methodical way of approaching a problem that you only really learn when you need to do proper science.

Another funny anecdote is during one of my classes, the professor would tell me “You’re thinking like an engineer, you need to think like a mathematician”. There’s nothing wrong with thinking like an engineer obviously - but having a different way of thinking about a problem, from first principles and rigorously, does offer a different tool in one’s toolbox.

And that’s really the gist. Now I’m back in production (temporarily) to get my paper into production and then I’ll be heading into my next project and finishing up the PhD with a 2nd publication!

I didn’t really want to start from some beginner’s tutorial so I did some hunting and found Arsen’s Vulkan streams which are perfect for what I wanted.

He blazes through the code explaining a lot of things, but with my DX12 experience, they’re quite brilliant because they get straight to the point and then I do my own digging into the spec.

This is the first time I’ve ever watched a coding stream and I’m really enjoying it. In my first ever development job in Malta, we did a lot of paired programming and this was ridiculuously useful. Seeing someone with 10+ years of experience coding (20+ in his case), making decisions and thinking out loud is very educational. One of the things that I like to hear the most about is not what they know about a topic but they learnt it - and seeing his problem solving approach is great.

Before I started it, I didn’t really have any major reason for leaning towards Vulkan other than curiousity, but at this point I’m pretty hooked, because of…

Validation Layers

The validation layers are tremendously helpful at telling you what might go wrong if you incorrectly setup certain structs. At some point I got a validation layer error that I wasn’t understanding, so I ran the debugger through the validation layer itself to see what was triggering it. Being able to see inside them was brilliant and a huge leap from the opacity that comes with DX12, albeit DX12 has a fantastic debug layer.

Mesh Shaders

Another major reason I was attracted to the streams was Mesh Shaders. They were a major reason for why I got my own RTX card, and having just finished the video where we get to do the mesh shading optimizations, I’m really looking forward to investigating more advanced uses for them. I’ve been fascinated with GPUs for a long time and I want to push my RTX to its limit with the amount of geometry it can process. Using the mesh shaders for doing extensive advanced culling is something I will definetly need when I try to process some of the massive scenes I intend to experiment with. The end goal is the Moana data set, we’ll see how that fares, I’m going to need a big GPU for that!

The main drawback is the lack of tools for debugging mesh shaders but more on that in the next section.

I’ll move the meshlet computation offline once I start working with bigger meshes because it can get quite expensive.

Debugging and Profiling Tools

It seems Vulkan is still not as well-catered as DX12 when it comes to debugging and profiling tools. I’ve had a lot of NSight crashes and it’s the only decent free GPU profiling tool I can find. RenderDoc is robust but of no help when it comes to mesh shading. I’m quite disappointed with the debugging and profiling tools for Vulkan so far.

It seems DX12 has Mesh Shader debugging support in NSight but not Vulkan. With DX12 we used Pix extensively and it is loaded with a ton of useful features like performance counters, BVH visualization, gpu buffer readouts. NSight crashed immediately when I ran my Vulkan program :( I’ll have to check my driver compatibility, hopefully it’s something as trivial as that.

volk

Extensions are an integral part of Vulkan, and volk comes a long way in solving that problem. It’s useful but I did the mistake of not reading the fine print in the documentation. I was including it into my own project, but I did not replace the vulkan.h include with volk.h inside the libraries I was including in my own source, which resulted in me chasing a missing symbol related bug which appeared as a simple access violation. This swamped a whole Saturday and Sunday, resulting in a cancelled pub session and some time with Dark Souls. Learnt my lesson now!

# Vulkan Spec

The spec is super-detailed and well written but whoever thought that it should be in 1 web page deserves a world of pain! If it wasn’t in 1 page I would consider it to being one of the finest pieces of literature, comparable to PBRT and Lord of the Rings :)

Reading the spec is very valuable, a lot of nuances are explained. The DX12 spec wasn’t this detailed (although it’s improved and I believe they’ve now released some really detailed documentation on GitHub).

Verbosity

Brace yourself for a lot of typing. In terms of verbosity it seems to go OpenGL -> DX11 ———–> DX 12 —> Vulkan. You have to write a lot of code to get things up and running. My previous experience with DX 12 prepared me for this so it was no shocker, but it can get exhausting.

Predictability

Using the Vulkan C API becomes very intuitive once you get used to the nomenclature. The validation layer guides you when you do something wrong, the structs follow a similar structure and so you kind of get used to the verbosity.

Final Thoughts

If you’ve never tried a low level graphics API, give Vulkan a go. It’s been a great experience (sometimes aggravating but what’s the fun in graphics programming if you sometimes don’t want to scream) and learning how the GPU works is something every programmer should do.

I’m loading the Buddha obj and rendering it using a mesh shader pipeline, returning the surface normal as a colour in the fragment shader. I’m using ImGui to display some stats, such as the FPS, GPU time and a plot of the GPU time. I store 100 samples and plot them, using the min and max of the samples to get it displaying well, otherwise it’s pretty much a straight light since the computation time doesn’t fluctuate much.

I’d be lying if I told you that you should look at my code, but you shouldn’t. It’s a horrible mess that’s been written in haste and hacky. But there is hope: I’ve also been watching Yan Chernikov’s Game Engine series which is reaaaallllllyyyy good and once I do a forward pass and a deferred pass in Vulkan, I’ll re-start the whole process and write th engine in a modular cleaner fashion. Unfortunately for now, all I have is a lot unclean code, but HEY IT WORKS SUE ME!

Here’s a pretty image of where I’m at:

With that, I’m signing off. If you have any questions feel free to reach out to me on Twitter or LinkedIn or email, ciao!

This blog post is the first in a series where I’ll discuss what I did to debug certain bugs that I encountered. The series isn’t planned - when I get a bug which gives me a lot of grief, I’ll write down what I did to find it and fix it.

I’ll preface that my renders look slightly darker than PBRT’s. I have many missing features such as RR, MIS, QMC sampling, all which affect this. I also have very basic sample filtering (just a box filter) and the renderer is not feature compatible with PBRT. Other than that though, it should look close enough to PBRT.

When I implemented my glass material and compared the rendered image to PBRT there was a huge difference between them:

This is the kind of bug that is quite difficult to debug - you have specific paths related to specific geometry which are misbehaving. Of course I knew where the problem, it’s somewhere in the Glass Material and Specular Transmission and Reflection BxDFs ( or so I initially thought :) ).

The first thing I started looking at was my Fresnel functions. Since that’s the function that determines whether a ray is refracting or reflecting, this made sense to me. I computed the function for certain angles and certain reflective indices by hand and wrote unit tests to confirm matches which they did.

With this out of the way, I looked at the refraction function and did the same - however everything was fine here too.

At this point I knew I needed to get in deeper, so I had to write some utlity code to be able to walk through a single pixel path, using the debugger and not render a full frame until I get to that pixel.

Coding it up didn’t require a lot of effort however working out what the ray was supposed to be doing required a lot of hand calculations. Another way to do this is to use an open source renderer and debug it, PBRT and Mitsuba are the 2 obvious suggestions.

Once I started up the renderer and had my first ray intersection with the glass, the ray was supposed to refract into the surface. The next bounce was going to be inside the glass and hit the other side of the box from the inside, and exit. My ray did no such thing! As soon as it was refracted inside the box, it refracted again but for some weird reason, it went back inside the box (which is actually a reflection). I had a hunch something was up with my normals, so I had a look through the code that might have been modifying the normals and realized that I was prematurely face forwarding my normals when I was populating my surface interaction record right after the first intersection.

I had not noticed this bug before because I was not transmitting rays through a medium for the Mirror and Matte material so it was never an issue.

Fixing that bug significantly improved my image

but this was completely different! I realized that having modified the normals during the surface interaction record population, another place where I was faceforwarding normals liberally was my Next Event Estimation code. I cleaned up this part of the code, rendered, got a match with PBRT and introduced reflection back in:

At this point I introducted the reflection back in but had a bug. I had been going at this for quite a few days (over a week), with only an hour or 2 to spare after work to look at this bug. So I took a bit of a shortcut and debugged a single ray through PBRT. This one I found with PBRT very quickly - I wasn’t setting the SpecularBounce flag to true when I was reflecting from a dielectric and so was computing direct lighting on a specular reflective surface. Fixing that gave me the final image:

Which is pretty much what PBRT is getting albeit slightly darker. One of the major pieces of advice I can give to people who are about to start their foray into ray tracing is the importance of unit testing. Testing every little function properly means that all the small moving parts work and gives you some peace of mind when using them and you’re hunting for a bug you might have.

I’m dedicating my August to just writing unit tests for my existing code - I was doing that for some stuff but in my haste to get some features up and running, I neglected that part and it bit me in the ass finally :) Having said that, I do look at bugs as a potential learning experience, and this kind of bug allowed me to gain a more solid understanding of surface reflection.

Another thing that was informative was following a path in PBRT. With a tool like the Visual Studio Debugger, this becomes quite easy.

One thing which I really want to do is render a single path, and in the mean time have my Vulkan engine draw the path and the normal of each surface intersection, and hovering over the intersection point, I can get the material details, the fresnel reflectance value if it was a specular surface, that kind of stuff. I have given it a lot of thought and would love to do this but time is a major obstacle.

That’s all I have today on this! The next post will probably be on debugging acceleration structures, the only issue is that I did the coding working on acceleration structures around February and as a result, my memory is a little hazy on my efforts so I might end up writing another acceleration structure to jog my memory and get new bugs, solve these problems and share the “wisdom”, with that, ciao!

I’ve been writing the blog posts for the next Ray tracing in One Weekend chapters, which I’m quite excited about, I’ve tried to be as pedagogical as possible.

I also have a post about how to go about building acceleration structures. In my opinion there isn’t really a gentle introduction on how to do this, and I’d like to fill the void. Just general tips and advice. I’m writing about my own personal process and the mistakes I made and how I fixed and debugged stuff.

The big blog post I’m planning for the summer will be about bidirectional path tracing. I’m going to provide a very simple framework and scene, for people and walk them through the theory and implementation. I’ve been thinking a lot about this and my aim is to simplify the concepts as much as possible.

On the work-side, we’re veeeeery busy with SIGGRAPH, but I’ll share later on what we’re doing, I don’t want to spoil it, but it is going to be brilliant! Tomorrow I’m giving a talk to the London GPU meetup group and I’ll talk about our initial efforts and investigations into getting some Realtime Ray tracing inside of Unity DXR. Whereas at GTC we kept it quite high level, I’ll dive deeper this time.

Chapter 2: One Dimensional MC Integration

Monte Carlo is widely used to solve integrals of higher dimensions, where an analytical method is not possible, or other integral computational methods are intractable. Dr. Shirley selects a simple integral to show how effective Monte Carlo integration is.

Start reading the chapter right until you get to the code that computes the integral. Don’t worry if you don’t get it, I’m going to explain everything.

The integral is \(\int_0^2 x^2 dx\) which will compute the area under the curve, with the limits 0 to 2. So how do we translate this into a Monte Carlo estimate? I was a bit confused by the sudden appearance of the 2 being multiplied with the average(x^2, 0, 2) function. Why are we multiplying the sample average by 2? Well as it turns out, this how we perform Monte Carlo integration. A generic integral of the form:

can be estimated with a Monte Carlo estimator by performing

\(x_i\) are uniformly distributed random numbers between 0 and 1, and \(N\) is the number of samples you pick.

Read the book right up until he proposes making his own PDF. I’ll lengthen the explanation and drive in a bit of probability, enough to show you the ropes but not enough to bore you (hopefully).

So the area function Dr. Shirley is trying to calculate is:

Knowing that \(p(r) = C'r\) this leads to \(p(r) = \frac{r}{2}\). The rest is pretty readable and understandable so I’ll move onto something else.

A short introduction into probability theory with a bit more formality will go a long way into helping you understand what Dr. Shirley computes in the book and other future graphics problems.

We’ve been talking about random variables for a while. We know that generating a uniform random variable means that we use a function to generate a number in a particular range and since we stipulated that it is uniform, this also means that any other number in that range could have been picked with the exact same probability.

We’ll introduce some terms.

• \(X\) represents the random variable. Since we’re using random variables to estimate something, this also is referred to as a random variable!
• \(P(X)\) represents the probability of \(X\).
• \(P(X = x)\) represents the probability that \(X\) is a particular value \(x\).

To exemplify these terms, in our statistical experiment we are going to toss a coin twice.

• \(X\) is the number of heads that we get. This is the random variable that is the result of our 2 coin tosses. \(X\) will either be 0 (\(TT\)), 1(\(HT\) or \(TH\)) or 2(\(HH\)).
• \(P(X)\) will be the probability of getting 0, 1 or 2 heads.
• \(P(X = x)\) will be a particular sequence of coin tosses, such as \(TT\) or \(TH\).

The cumulative distribution function (CDF) and the probability mass/density function (pmf/PDF) are the 2 pillars of understanding the nature of a random variable. The reason I used mass/density is because mass is used when we are describing a discrete distribution, and density is used in the continuous case. In this example, it’s the pmf (a coin toss is a discrete experiment) that I’ll be talking about, however the same ideas hold for the PDF.

This is the CDF of the experiment above:

As you may have noticed, the CDF is maxed out at one. 1 is the upper limit of the probability that an event from an experiment occurs - 0 is the lower limit. It is called cumulative because the probability distribution accumulates with each event. If you haven’t understood what that table is representing, let’s use our words!

• \(P(X=0)\) means that that probability of getting no heads is 0.25.
• \(P(X<=1)\) is \(\)P(X=1) + P(X=0)\(\). \(P(X=1)\) is \(TH\) and \(HT\) and therefore we’re adding 0.5 to the probability of getting \(TT\) which is 0.25, hence 0.75.
• \(P(X<=2)\) is just the probability of \(HH\) (0.25) + the previous 0.75, our final result being 1.

This is what the CDF looks like when we plot it

So can you guess what the PMF will look like? Hint: the pmf will describe the probability of each occurring coin toss .

If you sum up all the probabilities, they’ll add up to 1. So the PMF is showing us the probability of an event occurrence and CDF is showing us the probability of a range of events occuring. I know I keep on repeating this statement, but it’s common to not fully grasp the difference between the CDF and PMF and be unsure of their purpose. If you look hard at both, see you can go from the CDF to the PMF and vice versa.

So what happens if we’re dealing with continuous cases? Well for starters, the PMF changes to a PDF. We are also usually presented with functions to represent the CDF and PDF. To find the CDF from the PDF, we integrate the PDF from \(-\infty\) to \(x\), where \(x\) is a placeholder variable.

I’m not going to show an example because at this point I think you should go back to the book but wait until this final concept.

We’ve mentioned that we have been generating uniform random variables using a PRNG. In many cases however, we would like to take these uniform random numbers and would like to make them follow some other kind of distribution. We want to transform them. The way to go about this is a technique known as the Inverse Transform method. The technique is very simple and very powerful: Let’s say we have a random variable \(X\) , that has a CDF \(F(X)\). We want to generate values distributed according to this distribution. The steps to do this are:

1. Generate a uniform random number u between \([0,1]\).
2. Find the inverse of the CDF of \(F(X)\).
3. We find X by computing \(F^{-1}(u)\). \(X\) is now distributed according to \(F(X)\).

The Inverse Transform method warrants its own blog post so I won’t go into detail about its intrinsics and its edge cases and so forth.

Go back to the book and come back right after you’ve obtained the inverse of the CDF.

We have computed a Monte Carlo estimation of an integral, and now we’ve computed the inverse cdf of a PDF, which means we can transform our uniform random numbers into the PDF that we selected. If you recall, we said that we draw our samples from a uniform distribution from 1 to 0 in our original definition of the Monte Carlo estimator. The PDF of this estimator is \(\frac{1}{1-0}\) which is \(1\). The more general Monte Carlo estimator definition is actually this:

Where \(p(X_i)\) is the PDF from which the samples are drawn. By selecting a PDF that matches the shape of the function that we are trying to estimate, we are weighting the samples so as to reduce the variance. This is known as Importance Sampling! Choosing a good PDF is an ongoing form of research. For our integral, the PDF Dr Shirley’s picked is \(\frac{1}{r}\).

For the moment, we’ve been drawing samples from the uniform distribution, but thanks to the inverse transform method, we’re now using these uniform numbers and transforming them to match the new PDF. We’ll then compute our original integral with these new random variables and divide by the new PDF.

Now would be a good time to finish the chapter.

If you feel more mathematically inclined, Alan Wolfe has a great blog post about One-Dimensional Monte Carlo Integration. He’s also got a plethora of posts that will have some common content with this blog post series. I would also suggest you have a look at PBRT Chapter 13 if you’re feeling brave!

If you remember, we spoke about stratification previously. Let’s take the original integral \(\int_0^2 x^2\). To stratify the points, we’ll divide the interval in 4 bins. If I am taking N samples, we are going to take \(\frac{n}{4}\) samples per bin, so let’s call this value \(k\). The Monte Carlo estimate using stratified sampling now looks like this:

\(x_1\) is a random number between 0 and 0.5, \(x_2\) is between 0.5 and 1.0, \(x_3\) between 1.0 and 1.5, and \(x_4\) between 1.5 and 2, giving us a total of \(N\) samples.

Here is a self contained program showing the stratified and naive Monte Carlo estimations.

#include <iostream>

int main()
{
	float simpleMcRes = 0.0;
	int samples = 1000000;
	float res = 0.0f;

	int bins = 10;
	int samplesPerBin = samples / bins;

	for(int i = 1; i <= bins; ++i)
	{
		float binStart = (i - 1.0f) / float(bins);

		std::cout << binStart << std::endl;
		for(int j = 0; j < samplesPerBin; ++j)
		{
			float x = 2 * drand48();
			simpleMcRes += x * x;

			x = 2 * (i + drand48() - 1.0f)/float(bins);
			res += x * x; 
		}
	}

	simpleMcRes = 2.0 * simpleMcRes / float(samples);
	res = 2.0f * res / float(samples);

	std::cout << "Simple MC: " << simpleMcRes << std::endl;
	std::cout << "Stratified MC: " << res << std::endl;

	return 0;
}

Stratification warrants its own post following some discussions with some friends. I wanted to show certain properties but felt that this post would be too long.

That’s pretty much it for this post, next up is Chapter 3 and 4.

If you have any questions and feedback, feel free to comment or contact me via twitter/email.

My journey through the book raised many questions, which Dr Shirley was very helpful in answering and explaining. Along the way I read many different articles and saw different videos to understand things. I am probably not the only person who needed help understanding things and won’t be the last, so I thought it would be a good idea to share the information that I gathered.

The format of this post is unconventional - you should have a copy of the book (ideally the PDF as it is the most recent updated) open in another tab. I’ll try to guide you as best as I can throughout the book by asking you to look at the book and come back when you reach a certain point.

These statements will be in bold and red!

Re-reading is also very important - it’s very hard to understand everything at a first read. Don’t worry too much if you don’t get everything immediately. These are not straightforward concepts so just keep at it. If you feel truly stuck, reach out to me and we’ll try to clear things up. With all that said, let’s start.

Chapter 1: A Simple Monte Carlo Program

The first chapter is very straightforward. Demonstrating Monte Carlo by estimating \(\pi\) is a prevalent computational statistics example.

The code snippet for this that is presented in the book is very simple and self explanatory. A point is generated inside a square and if it is inside the circle, we increase the counter tracking the number of points inside the circle. We divide this final count by the total number of points we generate to get an average answer to \(\pi\).

It’s time to go look at the book - read up until the you get to the 2nd code snippet, involving the running \(\pi\) calculation program.

One of the questions of those uninitiated in probability and Monte Carlo theory might be: How does this type of computation work and how does it end up giving us an estimate of \(\pi\)? This stems from what is known as the expected value.

A classic example in explaining the expected value is trying to answer the question:

What is the probability that I will get a heads, when I flip a fair coin?

We know intuitively that it is 0.5. However what the expected value theory shows us is that if we

1. get an actual fair coin,
2. start flipping it,
3. take note of each result,
4. count the number of heads and tails,
5. average them,

We will probably get a result close to 0.5. The more times we toss the coin, the closer to 0.5 the answer will be.

The mathematical notation used to describe the expected value is:

• \(E[X]\) is the expected value of \(X\), our statistical experiment.
• \(x_i\) is the result of each sample that we compute of our function
• \(p_i\) is the probability weight of this sample. They must sum up to 1.

I’ll use another example rather than a coin toss to tie with the above equation. If we had a fair 6-sided dice, the expected value would be:

where \(x_i\) was the dice roll and $$ \frac{1}{6} was the probability weight. If dice wasn’t fair, the weights would be different - that doesn’t matter as long as they sum up to one.

Back to our original monte carlo example, the reason why this works is a theorem known as the Law of Large Numbers. The weak law of large numbers states that if we take a number of samples and average them, it will PROBABLY converge to the expected value. The strong law of large numbers states that it will converge ALMOST SURELY (with probability 1). We’re not going to go down this rabbit hole, the one of interest to us in the weak law.

An important term that we should define is variance. In our example, we were trying to compute \(\pi\), which makes this the mean we’re trying to estimate. The variance, in very informal language, measures how spread out our sample computations are from this mean. The lower the variance, the less spread out the samples are and the more accurate our computed mean is to the expected value (the expected value is also known as the mean). If we have high variance, this means our samples are spread out, and we might be using an ineffective function to computationally find our mean. There are ways to quantify variance which are useful for estimating the error of our Monte Carlo techniques, however I think it would not be fruitful for now to discuss those.

The Law of Diminishing Returns explains the idea that the more samples we take, the more we need than before, to get an accurate answer. In terms of numbers, if you want to half the error that you’re getting, you’ll need to quadruple the amount of samples. Here is how the error between our estimated value of \(\pi\) and actual \(\pi\) decreases as we compute more samples.

Notice how after sample 200, the error doesn’t decrease that much. This doesn’t hold a lot of significance for now but it’s good to keep this in mind.

Now would be a good time to go read the rest of the chapter.

Stratification is a way of intelligently placing your samples to estimate your function better. If you look at my blog post about random numbers, I show that a good PRNG (Pseudo-Random Number Generator) should generate uniformly distributed points. When you’re dealing with multiple dimensions a PRNG’s inherent randomness isn’t enough. Think of dimensions this way: you need 2 uniform random numbers for your primary ray, then 2 for your lens, then another 2 for your next bounce and so forth. Everytime, you’re adding another dimension and although your random numbers are unifornm if we look at them at being in the same dimension, it may not necessarily be the case when used in these different dimensions. To ensure true uniformity, without having clustered samples, we stratify the samples we take.

The careful selection of samples to minimize variance is an ongoing area of research, that is an absolutely fascinating subject and one of my favourite topics in CG. Dr. Shirley expands on this topic is in this blog post: Flavors of sampling in ray tracing. Leonhard Grünschloß has some excellent implementations of different Quasi-Monte Carlo based samplers.

I’ll slightly re-visit stratification at the end of Chapter 2, once we define the Monte Carlo Integral and importance sampling. The topic warrants tons of blog posts, maybe someday…

The next post will focus on Chapter 2, where I’ll delve ever so slightly into Monte Carlo theory and we’ll perform some interesting computations.

If you have any questions and feedback, feel free to comment or contact me via twitter/email.

I’ll assume you have working knowledge of calculus and some probability and statistics.

The most essential aspect of any form of monte carlo method is the generation of random numbers. Uniformly distributed random numbers on the interval [0,1] are what we’d like to generate first, as they are also what is required for the other types of distributions.

Generating random numbers using a pseudo random number generator (PRNG) means that we generate a sequence of numbers, using some particular seed value (some arbitrary constant) that initializes the random number generation algorithm. We use the term pseudo, because the numbers generated are not truly random but generated deterministically and hence can be replicated if the same seed and the same constants in the function are used. This is ideal for us, so that we can replicate and debug our algorithms. We can think of these algorithms as generating a deterministic sequence which is based on a starting point.

For this sequence to be considered to be of good quality, it should have certain properties, such as lack of predictibility (i.e. if we generate x values, and we can guess the next, this means the predctibility is possible) and equidistribution. For example if the mean of a large sequence of uniformly distributed numbers in the range [0,1] is not 0.5, then it is likely that something is wrong with the PRNG, given that due to the Law of Large Numbers, we would expect the arithmetic mean of this sequence to be similar to the expected mean (the expected mean being 0.5). We won’t focus on hardcore evaluation PRNGs and leave that to other minds, some useful links are “How random is pseudorandom testing pseudorandom number generators and measuring randomness” and TestU01 tests. We’ll look at 2 PRNGs and do some simple tests to measure their quality.

Let’s look at a simple PRNG known as the Linear Congruential Generator (which I’ll now continue referring to as LCG).

The LCG is an easy-to-implement PRNG, the function definition being:

where:

• m is the modulus constant, and m > 0.
• a is the multiplier, and 0 < a < m.
• c is the increment, 0 <= c <= m.
• X₀ is the seed value, 0 <= X₀ < m.

The above constants are all integers. Careful selection of these values ensures that the sequence we get is of a high quality.

Here’s an R listing of what the code for this should look like:

lcgen <- function(x0=4, N, a=1229, c=1, m=2048)
{
  x = c()
  x[1] = x0
  for(n in 2:(N+1))
  {
    x[n] = (a * x[n-1] + c) %% m
  }
  return (x[2:(N+1)]/m)
}

N is the number of random numbers we want to generate. Let’s generate 1000 samples and plot a histogram to look at the distribution.

It’s fair to say, the distribution of samples looks good. Given that the generation of a value is dependent on the next let’s see if we can see a pattern by plotting them in a time series.

That looks pretty random, so let’s plot them in series, taking pairs as cartesian coordinates (x_2k-1, x_2k). This type of test is known as autocorrelation.

Well that’s interesting, pairs line up as lines… what if we plot them in 3D?

The numbers are forming a line in 2D, a plane in 3D and this pattern continues in n-dimensions, where a n-dimensional hyperplane is formed for the n’th dimension. George Marsaglia identified this issue in 1968 and it is now know as the Marsaglia Theorem. He pointed out the worrying conclusion that all the papers that had used a LCG as their PRNG of choice might have wrong results. This highlights the importance of using a good PRNG.

From now on we’ll use a PRNG of a higher quality, the Mersenne Twister. I won’t go into its implementation, it’s a lot more sophisticated and robust than the LCG. The more recent PCG PRNG is better than Mersenne and it’s what I use in my rendering system. Having said that, R’s runif (short for random uniform) command uses the Mersenne Twister algorithm, and for these posts it should suffice. PBRT 2 used to use Mersenne Twister and PBRT V3 uses PCG.

If we re-run the above tests using Mersenne Twister, we can see that, the histogram looks acceptable,

so does the time series plot,

and the 2D plot (points look to be well distributed in the 2D plane)

and the 3D plot (points look to be well distributed in the 3D unit cube).

Whenever you see runif, assume the numbers are well uniformly-distributed using the Mersenne Twister algorithm. Now that we’ve understood a bit more how uniform random numbers are generated, I’ll later post about transforming these uniform random numbers into other forms of distributions, e.g. normally distributed.

I’m at a point now where I am importing meshes of a larger nature, and I was not content with my BVH implementation. As much as I would like to read the state of the art papers on BVH construction and traversal and implement them myself, I’ve decided to opt to use embree instead. I would rather focus on the light transport part and focus on getting prettier images and getting them to converge quicker from an algorithmic standpoint. As a friend of mine said, I have to pick my battles, and embree is going to do a lot of heavy lifting for me so that I can focus more on what I’d like to write.

Setting up embree with CMake is quite easy I’ve got the full source code of this post available here , and I won’t go into details into how to set it up - the CMakeLists.txt is quite self explanatory and I’ve added as much comments as I can.

As great and as fleshed out the samples are, there is a lot of boilerplate code, written in different header files. This put me off initially but with a bit of digging, and building the samples with a powerful IDE went long way in help way in helping me zip around the API and the headers.

I thought it would be interesting to write the simplest thing I can think of with embree, a hello world ray tracing example if you will, I’ll explain below what we’ll do.

What is embree?

Embree is a high-performance ray tracing kernel library written by Intel. It contains state-of-the-art implementations of acceleration structure and intersection methods, and provide a plethora of features to write a performant ray tracer. More details can be found here, and in their white paper. They target the Intel architecture specifically and are highly optimized using SIMD intrinsics to push as much as performance as possible.

Hello Embree!

My toy example shoots a ray per pixel and returns white if nothing is hit and black if the triangle places in the centre of the image is hit. This is the main function:

int main()
{
    // Inititiate device and scene
    // Add geometry to scene
    // Initiate image parameters, image and camera
    // Render image
    // Output image
    // Release scene
}

We’ll work through each of these with a short explanation for everything. The full source code listing can be found here. It is self contained, with embree being part of the sources. If you have any issues with compilation, feel free to contact me.

If you want to skip ahead:

1. Device initiation
2. Adding geometry to the scene
3. Creating the triangle geometry and defining the vertices
4. Render image
5. Output image
6. Release scene

Device initiation

// Inititiate device and scene
RTCDevice device = rtcNewDevice("");
RTCScene scene = rtcNewScene(device);

The device object is defined to be a class factory for all the other objects that we’ll be creating such as the scene and the geometry, using the rtcNewDevice call. The device handle is not destroyed until all objects bound with the device are released. Embree uses reference counting to keep track of the lifetime of all the objects you create - functions with the word release decrease the reference counter. It’s very important to use the release functions, otherwise you’re leaking memory. I’ve heard that for every byte of leaked memory, a kitten drowns and we don’t want that now do we?

The new scene is created and bound to the device previously created.

Adding geometry to the scene

We now need to define and add the geometry to our scene. I’m just going to create a triangle in the middle of the screen.

// Create a new geometry for the triangle
// Define the vertices
// Assign the vertices to the geometry buffer 
// Commit geometry to the scene

Creating the triangle geometry and defining the vertices

We’ll need to allocate memory for our geometry:

// Create a new geometry for the triangle
RTCGeometry mesh = rtcNewGeometry(device, RTC_GEOMETRY_TYPE_TRIANGLE);
Vertex* vertices = (Vertex*) rtcSetNewGeometryBuffer(mesh, RTC_BUFFER_TYPE_VERTEX, 0, RTC_FORMAT_FLOAT3, sizeof(Vertex), 3);

We create a geometry object using rtcNewGeometry, which is attached to our device and we’ve picked a triangle to be our geometry of choice. Embree has a selection of different geometry types available, which you can find inside the documenation. Once we’ve created our geometry object, we create a buffer for the vertices, using rtcSetNewGeometryBuffer. We’ll use this function to set the indices later on. The key here is the second parameter, RTC_BUFFER_TYPE_VERTEX.

Next is the vertices:

// Define the vertices
vertices[0].x = -1; vertices[0].y =  0; vertices[0].z = -3;
vertices[1].x =  0; vertices[1].y = -1; vertices[1].z = -3;
vertices[2].x =  1; vertices[2].y =  0; vertices[2].z = -3;

Nothing extraordinary here - just defining the positions of the vertices. We’ll now assign these indices to our buffer:

// Assign the vertices to the geometry buffer
Triangle* triangles = (Triangle*) rtcSetNewGeometryBuffer(mesh, RTC_BUFFER_TYPE_INDEX, 0, RTC_FORMAT_UINT3, sizeof(Triangle), 1);
triangles[0].v0 = 0; triangles[0].v1 = 1; triangles[0].v2 = 2;

We’re now setting up an index buffer for the vertices. Looking at this from a birds eye view, it’s quite similar to how things are done in OpenGL. We’re not done yet though. We need to commit the geometry and the scene.

// Commit geometry to the scene
rtcCommitGeometry(mesh);
unsigned int geomID = rtcAttachGeometry(scene, mesh);
rtcReleaseGeometry(mesh);
rtcCommitScene(scene);

So what does this all mean? The rtcCommitGeometry call commits all the modifications to the geometry and has to be called each time we modify the geometry.

We then attach the geometry to the scene using rtcAttachGeometry. The rtcReleaseGeometry call decreases the reference count of each piece of geometry. I was quite curious when I saw this and I think it’s because rtcAttachGeometry increase a reference to the mesh by the scene, so the original mesh handle can be disposed of. To commit our scene we’ll need to call rtcCommitScene which builds our spatial data structure. Everytime we modify the geometry, we need to go through this step. We can commit to the scene all once though, as you’ll not might notice in my actual source code that the commit is right after the call to the function that defines the geometry. You can define and commit all your geometry at once and then have a single called right after these definitions.

We’ll initiate the image and the camera parameters.

// Initiate image parameters, image and camera
const int width = 600;
const int height = 300;
std::vector< unsigned char > image;

embree::Vec3fa lower_left_corner(-2.0f, -1.0f, -1.0f);
embree::Vec3fa horizontal(4.0f, 0.0f, 0.0f);
embree::Vec3fa vertical(0.0f, 2.0f, 0.0f);
embree::Vec3fa origin(0.0f, 0.0f, 0.0f);

If you’re not familiar with regards to the parameters of the camera, I’d suggest reading up on primary ray generation and the camera model. I might do a post about this later as I used to struggle with this a while back. Now that I’ve said this in public I’ll be forced to do it :)

Nothing too fancy here, just define the image parameters and the camera parameters for the primary ray generation.

The next part is where we get to test the ray intersection!

Render image

// Render image
for(int y = height-1; y >= 0; --y)
{
    for(int x = 0; x < width; ++x)
    {
        // Initiate pixel colour
        embree::Vec3fa color = embree::Vec3fa(0.0f);
        // Create ray intersection context
        // Generate ray
        // Perform ray intersection
        // Check if ray intersected any geometry
        // Set pixel colour
    }
}

We’ll now define the intersection context:

// Create ray intersection context
RTCIntersectContext context;
rtcInitIntersectContext(&context);

We use the RTCIntersectContext for your ray intersection test. rtcInitIntersectContext takes flags which can optimize the performance of the ray tracer, such as indicating that the rays are coherent.

This is the primary ray generation stolen from Dr. Shirley’s Ray Tracing in One Weekend book.

// Generate ray
float u = float (x) / float (width);
float v = float (y) / float (height);

embree::Vec3fa direction(lower_left_corner + horizontal*u + vertical*v);
Ray r(origin, normalize(direction), 0.0f, embree::inf);

Once we have the ray set up, all we need to do is perform a ray intersection test using an embree function suited for single ray intersection tests.

// Perform ray intersection
rtcIntersect1(scene, &context, RTCRayHit_(r));

We need to pass in the scene and the context. The RTCRayHit_ function constructs the embree ray-type from the one that type that we defined ourselves. In reality our ray is very similar to what’s the defined in embree. The intersection information is stored in the ray, and checked:

// Check if ray intersected any geometry
if(r.geomID != RTC_INVALID_GEOMETRY_ID)
{
    color = embree::Vec3fa(1.0f, 1.0f, 1.0f);
}

and that’s it! The embree specific stuff ends here. I haven’t included the ray definition and the auxiliary functions to convert because they’re one liners and you can have a poke around in the repo. We then set the colour of the pixel:

// Set pixel colour
image.push_back((unsigned char)(color[0] * 255.0f));
image.push_back((unsigned char)(color[1] * 255.0f));
image.push_back((unsigned char)(color[2] * 255.0f));
image.push_back((unsigned char)(255.0f));

and repeat the whole process for each pixel. When that’s done, we spit out our png:

Output image

// Output image
unsigned error = lodepng::encode("hello.png", image, width, height);
if (error) std::cout << "encoder error " << error << ": " << lodepng_error_text(error) << std::endl;

I’m using lodepng to write the image to file, nothing too complex going on here, and then we get this!

Release scene

Now we should clean up after ourselves and leave no memory leaks

// Release the scene
rtcReleaseScene(scene);

and that’s pretty much it.

Thanks for reading, I know this was quite verbose but I hope it was helpful. Embree’s extensive features means that it’s now significantly easier to write a high-performant ray tracer. I’m very excited in doing more fleshed out examples and melding it with my own ray tracer. I will have to probably change a lot from an architectural standpoint but the performance gains warrant it. If you have any questions feel free to ask, I may not necessarily know the answer but learning is part of the fun and will do what I can to help :)