Wednesday, July 30, 2008

Do we like bugs?

Cus I have a lot of em! The fixes for the surfaces had lots of bugs. And now they are fixed. I don't have much time so this will be brief.
  • Metaball ratio now works. So rather than changing spacing in the advance menu you can just set this to a value larger than 1. You will then need to adjust the threshold and if you get funny ray traced isosurfaces, you need to put the max gradient up to about 3 or higher as needed. This can slow down raytracing a lot.
  • Doubling the metaball size will make render about 8 times slower. Use the sampled grid if you can.
  • You can now convert your physics object to a triangle mesh. Just how legal the mesh is and how well it works in the rest of AoI is prone to ... unstable behaver.
  • I completely revamped the synchronization code using ReadWriteLocks. Seems to work well, but I still cheat a little.
  • This means there should be no more flashing animations. If you do get one with this new version let me know.
  • When you reload the file you worked on "yesterday" the particles are duped. This means that the first frame from you bake is stuffed. Don't use it to see whats going on and don't "rebake" from the first frame without reseting the whole thing.
  • Generally if something seems to start to go wrong. Reset, Rebake.
You know where to download it from.....

Sunday, July 27, 2008

Surface quality and another beta-alpha release.

I have been spending some time on working out how to get a nice looking surface. As you can see from previous posts, the surface currently looks rather "blobby". So I have read papers and done some experiments. I think I am getting somewhere, but there is much more to be done yet.

First of all I now let the surface kernel be different from the SPH kernel. The SPH kernel has a lot of math that it must obey in order to work properly which a "viewing" kernel does not need. The result is a much nicer looking surface, but far from perfect. There is a catch, you need to set the isosurface threshold down to 1 rather than the default 500. Otherwise you won't see a thing. My current viewing kernel is (1-r^2)^2 and is often used with metaballs. The results is much better and just results in a small amount of roughness. With some adjustment even better results are possible.

Here is an example:

With a larger threshold:

Both in a composite:
Both together

Same but with polygons Ray traced:
Ray Traced Poly Surface

lower threshold and higher poly res. Note that rendering time is dominated by the mesh build. So this could be much faster for transparent fluids.
Ray traced with finer polys and larger threshold

And a composite:
The Above together in white this time

As you can see the threshold can make some difference. Generally there is no right value and it will need to be adjusted. It should be noted that other fluid packages has similar parameters that need to be adjusted. Surface tracking is not a exact problem and hence the different methods that are used to tackle the problem. Also this will make less difference with very small particle sizes. But it will be slower.

The final version will be almost be 100% based on a mesh. But a good mesh is very slow to build and hence may need to be backed in with the rest of the simulation. In real flow you can bake lots of mesh's to a single "particle" bake and you can then adjust the different parameters better for different views (aka close ups vers far shots...).
The trick with meshes is not that they are better, in fact they will be the same as the isosurface examples above. But note that these examples are only a little rough and some smoothing could make it almost perfect. And that what we do. We add post smoothing to the mesh, usually via a iterative scheme that can be tuned.

Unfortunately the mesh code has some issues. First is the fact that there are bugs. For some reason when the mesh preview is on the cpu goes to 100% and never stops. Has anyone else noticed this? The next is that there are some cases that i get null triangles. And the biggest is that is uses some code that nobody seems to have written. This creates clear issues with the GPL'ness of the code base and I may just have to rewrite it.

I will still keep the isosurface code and at this point it gives the best results. You get some performance issues however and the best way to deal with that is to turn on the useSampleGrid option. In my scene above they caused the render to be about 5-10 times faster, and the difference would be even greater with transparent fluids.

Other changes are there are now some options for fluid objects. The first is a dither and just randomizes the particle positions so as to loose perfect alignment effects. The other is x, y and z ratios. These are fill ratios in these respective directions. The default gravity is in the y direction so if we set y to 0.5 the object will be half full. Also the fluid objects and softbody objects no longer get "blobs" at the vertices's.

Add one more render:

The updated plugin is at the same link as last time. The last post still makes up some of the release notes.

Do not forget to change the threshold to about 1 or a little more for this version.


Wednesday, July 16, 2008

The Beta Alpha ++ release.

Yep a new release is here. First some movies:
loopable AoI
Filling an A with the old boundary
Filling an A new boundaries

There is a lot in this release so we will start with a list derived from the change logs.
  • Fluid objects are in. They are not "stabilized" and look rough. But we can now start with liquid in the scene.
  • New boundary parameters have been added and generally work better than the velocity bias parameter. The parameters are a factor to increase boundary force and a clamp value to limit the max value of that force. If the clamp value is greater than one particles can bounce a bit.
  • All particle boundary code has been more or less removed.
  • The integrator was made more consistent. This mean one more pass per iteration, but is more stable.
  • Softbodies are working.... Kind-of. WARNING, this can break your scene, have a backup.
  • Note that you still need to subdivide emitters.
So the fluid objects do work pretty good at this stage. They are however rough in that the surface does not look so good. When stabilization is added it will improve. What is stabilization. Say if you have a block of fluid and you place it into the same block as a boundary. When the simulation starts the fluid will move a little in order to settle properly. Its not a big deal but a "lake" will drop a little over the first few frames/seconds of the animation. Don't forget when using small particle sizes that you also need to drop the time steps as well.

Next is the boundary code. I'm a lot happier with these parameters to control boundaries than the velocity bias. The boundary factor is just a simple multiplier on the base force and clamp simple clamps the value. So the default is that the boundary has a effect of one on the surface. If we set the multiplier to say 1.5 and leave the clamp at one. Then the "thinness" of max effect is now larger. But the magnitude of the maximum effect is the same. If we let clamp be larger than one then the boundary gets a little spring to it and can push particles back rather than stop them from traveling though the boundary.

But what about softbodies! Well first of all its a long way from finished. Ok so not that long. But this really is a alpha alpha stage in that I does delete data from the scene and generally does not like anything other than full rebakes every time. So here are the guidelines:
  • Don't use high poly/vertex count objects.
  • Keep the spring stiffness high. About 5000-100000.
  • If the object collapses the simulation will go unstable.
  • If you want fluid interactions, then you will need to up the mass values to more than 1 on the softbodies. Also smaller time steps will be needed.
  • Every time you make a change to the scene rebake.
  • Once a object is set to softbody its animation tracks may get deleted. You cannot move the object around the scene. Reload the scene and position the object before adding it as a softbody.
  • Vertex are mass. If you have a lot of vertexs at one end, thats the heavy end.
  • avoid long and short vertexs in the same model.
So after that list does anyone want to try it? Note that softbody objects must be triangle meshes and can have approximation/interpolation smoothing.

Now I often get complaints with the large number of options. That is the deal at this stage. I/We need to be able to work out what works and what does not. The only way to do that easily is have the options available to try in different situations.

Happy AoI'ing....
Oh, I almost forgot, the plugin is here:
The Plugin

Sunday, July 13, 2008

Softbodies and soft bugs...

In the interests of interest, I have been working on the softbody section of code in the fluid plugin. Thats right, its called Physics.jar for a reason. The entire structure and methods used are there so that fluids, softbodies and rigid bodies can be integrated into one single engine. This will allow fluids to interact with these objects. But it does mean less optimized "sub solutions".

Currently the softbodies are working, with some known shortcomings while I sort out the finer details. Also I keep finding bugs. Right now there needs to be a little bit of a clean up of the raw simulation code. I have too many methods that don't properly document the contract for the engine. So boundaries are a bit of a interesting mix of problems really. Its all experimental code at the core and thus its far from clear what methods work well in practice. A good example is the no slip boundary conditions that are frequently used. When you are using fixed mass particles this turns out to be a really bad approximation, as the boundary layer is smaller that the particle resolution. Simple friction with velocity normals work very well and are much more stable.

I have now read more papers than I care to count on the subject of SPH. There is a general pattern however, and that's that its generally about a 1st to 2nd order method at best. Boundaries are where the side is let down, and even thou there are corrections (that I use) the smoothing kernels limit the local approximation across the parameters. To make a long story short, I'm not aiming for a 3rd/4th order method. In fact I estimate that I have 1st order at the boundaries and 2nd order throughout the fluid. I use only first order explicit time integrators. The combination turns out to not be so bad after all. So I keep the code that works, and its a little adhoc right now.

So the current softbody list of features and antifeatures:
  • Interact with each other
  • Interact with fluids. ie can float
  • Must be a triMesh
  • the mesh can be set to approximating to smooth deformations.
  • Interactions with each other is from vertex to face. So few vertexs means they tend to interpenetrate a bit.
  • Unfortunately you need quite small time steps to keep stable because of the explicit integration scheme.
And a very small example...
Soft body and fluids

Monday, July 07, 2008

The first Alpha Beta release....

The final dusting to the poly boundary code have been completed. So now you can play with the new code. There are a few extra options now for boundary's that I will try to explain here. At this stage its about making options avalible so that others can play with setting to what works and what does not.

The first new option is the PolyBoundry. This makes it a polygon type boundary rather than a vertex based boundary object. The remaining options don't work unless PolyBoundry is true.

The next option is the calculate boundary outside poly. What this means is does a polygon effect extend out from the edges rather than the projected normal volume. In more technical terms, if we project a particle with the plane normal onto the plane that contains the polygon, do I do anything if its outside the polygon. Generally setting this to false works best. But for convex shapes there is the possibility of "leak vectors" . Setting it on mean that concave sections give too much boundary force.

The last new option is really for the case where Calculate boundary outside is set to false. Generally we will get very slow leaks over time. If we provide a constant velocity bias to the boundary conditions we help move particles that are close to leaking, back inside. But this is adding energy to the system and therefore can lead to pretty weired results. Below are some examples, but this should be small 0.01 or smaller. Always try 0 first. Even at low values that can make still fluids "boil".

There are some limitations to the current implementation. The biggest is the complete lack of friction. All current boundary's are frictionless and that can lead to bizzar results from what you might expect. Another is getting fluid boundaries to match the boundary surfaces for rendering. Currently there is no way to ensure that ray tracing a glass with water will be done correctly as far as I know. Its also not currently optimized either meaning that its slow that it needs to be. But its still faster than particle boundaries. The last limitation is the need to put the boundary triangle into all voxels that it overlaps. This means that using smaller particle sizes will increase memory requirements for the boundary objects.

So here is an example of the unstable rotation that is now, hopefully fixed.
rotator mpeg 4

This a example scene with the outside boundary's set to true.
Hour Cup
Note the large distance between the walls and the particles.

The same thing but with outside boundary's set to false.
It now leaks, but only slowly.

Then we add just a little velocity bias (0.01).
bias fixes leak
We can really see the lack of friction here.

Getting stupid with bias (0.05).
Too much bias
Now the lack of friction is compounded with the extra energy added by the boundary.

Finally the file I have used.
AoI example file

This plugin is now only for AoI 2.6. But only because I don't use 2.5. Also its a beta-alpha for a reason.

Download the Physics Plugin

Sunday, July 06, 2008

Polys for boundaries and stupid bugs

Oh yes, Its finally done. We have polygon boundaries so you never need to subdivide a mesh again. This should make it about 200 times easier to use now and I will get much less over/under subdivided meshes, asking where it all when wrong.

Ok, so its not finished. The source objects only use vertices's, and we still need to convert to triangle meshes and you still can't have a initial condition that includes a full glass of water. But otherwise this is working with the only regression I have noticed is the save state issue discussed below.

However there was one bug that took almost the whole weekend to find. When I had a object rotating under a fluid source, it would go unstable. Even with time steps down to 1e-6 it would still go unstable. To make a long story short I did not really find the bug. But i did prove that it *only* occurs on perfectly axis aligned vertices's rotating in the x plane! Even rotated .1 deg in the y axis would prevent the problem. I think I fixed it but I'm really not sure. Its possible that you just need everything to be perfectly aligned in some other axis....

But its good enough for me. So I will package this up on Monday as a beta-Alpha release for those who want to give it a try. Remember that performance on the old highly subdivided meshed will be very poor at this point in time, so a "clean" slate may be best. Also remember that there is no guarantee that there will be any level of AoI file backward file compatibility.

Also I will update the blog with a link to some new movies Monday as well.

Wednesday, July 02, 2008

Real Triangles, well infinate planes anyway...

The first big steps to removing the need to subdivide meshes has been taken. The biggest chunk of the required infrastructure is in place and working. The triangles however are approximated as infinite planes instead. But due to the fact that I use the particle boundary code there should not be to many issues to iron out as most of the code has been used a quite a bit.

I make one quite big assumption. I approximate all properties across the face of a triangle as a linear combination of the vertex values. This means that velocity of the face is not accurate for large polygons that are rotating, but this can be solved simply with subdivision for these perhaps quite rare cases. Another consideration is that the force of the fluid is transfered to rigid bodies in a similar fashion for rigid body dynamics, and this could also lead to inaccuracies and perhaps instability.

The code is not in the best state, and there is still quite a bit of try and comment out code as well as out right hacks in there. The is already one regression. The plugin does not preserve state of object that are created after the physics plugin for example.

In order to catch regressions early, I was thinking of releasing regular snapshots that are simply totaly untested. If folks would like this it would mean that regressions would be picked up sooner rather than later. Otherwise the whole debug cycle gets a little slow. Let me know what you think.

So the next *proper* beta release (0.7) I will hopefully have the following fetures.
  • Full polygon boundaries for moving and stationary objects.
  • Basic Rigid body dynamics.
  • Animating parameters.
I have also noticed some serious performance issues. Its just much slower than it should be. This will perhaps not get looked at in the next release.

Sunday, June 22, 2008


In the interests of getting work done when I'm on a train, I just got myself a new laptop. This is in fact my very first laptop and unfortunately it has been cursed with Vista. But I already have cygwin working and will put Linux on it once I have at least some good uptime with a live DVD.

In the mean time this means that it will be much easier for me to work on things like the fluids plugin when I'm away from my fixed machines.

Monday, March 24, 2008

Other Things

This is just to give a update on what i have been doing. Although it was not on the AoI fluids plugin. But it is not forgotten, and if i ever do abandon it, I will release a full source version. Remember I want this plugin too. Your feed back was read and will be reread, so its not wasted.

So what have i been doing. Well I been to New Zealand and back. I have got a contract in the wings that means I can stay in Austria for another 2 years (maybe I will learn some German?). I have noted my lack of ability to Get Things Done, found a book with that title and I am finding it really helps to, well get things done.

Last but not least I have migrated all my code from subversion to Git. Oh what a joy git is compared to cvs and svn.

Its interesting how I found out about Git. I started using CVS for everything in about 1995-7 (can't remember exactly as I cleaned and reset my repository in 2000). I was not that impressed. I could not do a lot of the things I wanted to like move files. Binarys was not well handled and some things were far slower than I thought they should be. I did try a merge from a branch once.... and only once...

Then I ended up working for some companies that used source safe. Now I hate working on windows at the best of time. Nothing personal, I just find that MS support and documentation is terrible compared to other companies I have worked with. But what the Hell was MS thinking with source safe? Surely they didn't use it in house? To make a bad tool worse, the company I was working with had very stupid polices regarding file locking.

To make a long story short, I fell in love with CVS because she was the prettiest girl at the party. Not because she was pretty.

I kept looking for a better way because there must be a better way. Hence I found subversion. I have now used it for about 3 years. The problem is that its not really a better way, but the same way. Combined with the tools in IDE's (eclipse mainly but netbeans as well) my repository kept getting into limbo states preventing commits. It was difficult to fix without a complicated rolling backup system. Further to that problem, which directly affected the fluid plugin, was that I would end up out of sync with home all the time (I had no internet).

It was the fluid plugin devlopment that got me thinking that my personal devlopment model was in fact distributed. It didn't take long to find git. Unfortunately the admin at work never would give me a time when he would install it at work. So i waited until our machines got upgraded which was last week. Now all my repositories have been transfered to git. I don't use IDE tools for version control anymore since they just don't work that well. But with git is does not matter because the checked out directory is clean.

So why Git? Here is my list.
  • Distributed development.
  • Fast, very very fast.
  • Merges work so branching becomes common. This means there is better history too.
  • Unix tool set.
  • Very good repository integrity checking. This is a big deal IMO.
  • Easy to use (edit) configuration file.
  • Frequent branching makes "frozen" release much easier to manage. Debugging on the frozen branch can be easily merged with the main devlopment trunk.
The is even some tools to import a subversion repository with history. So I still have a history back to 2000. In short, even if you are a solo developer I would still recommend Git first and distributed version control over everything else.