Dev:2.4/Source/Physics/Particles Rewrite - Blender 2.46

Based on my previous thoughts and ideas on particles' future I've started a full rewrite of the particle system code.

Getting the particles

No more patches. As of 17.6.2007 my particles are in a separate branch and can be checked out with svn from https://svn.blender.org/svnroot/bf-blender/branches/particles. I'll try to keep it in synch with trunk on a weekly basis, but this is all really new to me so let's see how it goes.

Please send all bugreports & ideas to jhkarh at utu.fi or catch me at blendercoders.

Documentation

I've started a documenting effort here in the wiki at Dev:Source/Blender/2.46/Particles Rewrite/Doc. Feel free to contribute if you have experience with my particles. And if you make mistakes I'll try to correct them :)

General ideas

Particle system step

For each positive change in time a particle system step is done where dynamics are calculated for particles based on their previous coordinates. For negative changes in time particles are reset to emit locations (like softbody does it).

Modifier stack

• each particle system has a ParticleSystem struct and an entry in modifier stack
• the modifier just stores a DerivedMesh for the particle system to use when it needs to
• the dm is updated and marked to be so whenever the stack is recalculated
• secondary function of the modifier is to allow explosion of the emitter with the caclculated particles, this requires that particles are calculated between the dm saving and exploding
  • edgesplit tries to split edges at particle influence boundaries for smoother shrapnel

http://users.utu.fi/jhkarh/media/code/particles/edgesplit.jpg

• the modifier also calls the particlesystem steps in between recalculating the dm and possibly exploding.. this way particles allways get updated dlm and explosion is done from updated particles
• as curves only allow deforming modifiers I had to modify the modifier stack a bit to allow modifiers with both deformVerts and applyModifier as defined functions thisway meshes can use the explode enabled modifier and curves can just store a dm created from the curve displists
• modifiers are initially named with the number of the particlesystem they belong to

Particle emission

• when the p.sys is created or if the dm has changed we calculate totpart amount of emit coordinates on the dm (for example face number & coordinates along 2 edges of the face)
• these coordinates are stored in the NewParticles
• when a particle is born the emit coordinates are converted to global space locations from the current state of the dlm and we can start dynamics calculation for the particle
• for curves faces & volume distributions default to verts distribution
• different ditributions:
  • verts - from vertices in order or randomly (for curves this is from displist vertices)
  • edges - from edges in order or randomly (for curves this is from the edges between displist vertices)
    • possible selection of distribution on a single edge (random, linear, ??), sill todo, now just randomly somewhere on the edge
    • also still todo, ditribution based on edge length to create even distribution
  • faces - from faces in order or randomly
    • possible selection of distribution on a single face (random, jittered, ??),sill todo, now just randomly somewhere on the edge
    • even distribution considers face areas to create less particles on small faces and more on bigger faces
  • volume - just like from faces but then pushing the particles in direction of face normals (still experimental)
  • interpolated - fast hair creation!
    • normal particles are created from vertices, these are used as "guides" for lots of more particles
    • user selectable amount of particles are interpolated between these guide particles at drawtime
    • allows fast editing & calculation of a lot of particles because only the guide particles need to be calculated
    • certain effects like path length & textures are still individually calculated for every interpolated particle
    • the guide particles are allways drawn fully but never rendered

Baking

Should be ok with same type of baking a softbody i.e. we start from a certain frame and step through every frame as long as wanted & store each step result. This might change though as Ton wants a GUB (Grand Unified Baking system :) for everything bakeable. I've now started implementing a unfied baking system which can be accessed from the outliner.

• It can now bake particles and softbody by selecting as many of those wanted in the outliner and by rightclick menu->Bake. By the same way the baked things can also be freed with menu->Free Bake.
• Also the outliner will show info on the bake like mem consumption etc.
• All "normal baking" for particles is done just by flagging the particlesystem "to be baked", and it will then be baked after updating objects through depgraph.
• Normal softbody bake is still yet to be done.

Ah well.. maby this whole baking will have to be thought bigger & better. So for now only crappy basic baking functionality in particles.

• Baking spaces
  • Global: normally through time with physics etc. in global space (used to be dynamic baking)
  • Object: baked in a single instant of time into object space, i.e. in dynamics all else in the scene is frozen in time and the particles are alive (used to be static baking)
  • Geometry: baked particles stay attached to the emission geometry when the emitter is deformed (for example with armatures) (replaces animated statics with superior performance & functionality)

Softbody integration

• static baked particles can have a softbody applied to them
• great for hair dynamics
• some tweaking of things to allow better specialization to hair is still needed
• works by selecting "psys" in softbody panel -> all things in softbody panel now apply to the current particlesystem, unselecting "psys" shows normal object softbody settings again

Bake editing

• allow editing of all baked paths in a dedicated sculpt/edit mode for path keys
• currently single keys can be selected & transformed mostly like mesh vertices
• scaling affects particle velocity
• single/multiple particle's/ particle key's timing can also be adjusted from the n-key panel
• editing can be done so that the path lengths stay constant (hair doesn't stretch)
• editing can also be done so that paths are kept from going through the emitter surface
• baking doesn't nescessarily produce nice keys for editing so selected particles can be re-keyed to have as many keys as wanted distributed evenly along the path without changing the paths shape very much
• currently coded editing brushes
  • cut
  • comb
  • grow
  • shrink
  • add
  • weight
  • smooth
• if the baked data is freed the editing is lost
  • not yet sure of how animated statics could work here
  • should investigate for a way to be able to retain editing between free bake - rebake

Visualization

There are two separate processes: One creates the particle data (the things explaned above) and the other interprets that data to allow visualization & rendering.

• Makes many things possible that would be very difficult by applying forces etc. to the particles when the dynamics are calculated
  • applying & modifying wavyness to strands after the strands are baked
  • curve guides can also in addition of normal dynamics time be applied at this interpretation stage to baked paths
    • much faster than dynamic guides as there's no baking in reaction to guide curve changes
  • showing any number of particles along baked paths
  • varying path length either relative to the whole path or using an absolute length in blender units (hair growth!)
• different interpolation types are needed for baked keys
  • cubic
    • takes velocity information in baked keys into account
    • great for particles but bad for hair since when you edit the paths the velocities can't be updated so that smooth paths are maintained
  • linear
    • can be usefull for special cases
  • cardinal & b-spline
    • only uses particle key locations to create the path
    • we loose a bit of controll but gain allways smooth paths == great for hair editing
• rotations need to be interpolated differently too (mainly usefull with children)
  • Keys
    • the actual rotation of the baked keys is interpolated along the particles path
    • gives poor results with hair made with children
  • Direction
    • rotation is determined by the difference of the current velocity to the velocity at the root
  • Incremental
    • rotation is determined by the small incremental differences of the velocities along the path from root to the current position
    • slowest of the rotation modes but gives best results for hair
  • Z: makes the rotation at root be aligned with the Z axis
  • I: uses the initial rotation at the root to calculate rotations along the path
• different draw types
  • Nothing - usefull for explode or when you don't need to actually see the particles but whant to do something else with them
  • Rendered as halos (usable to distinguish different particle systems' particles in viewport
    • Point - normal points (old Blender particles visualization)
    • Circle
    • Cross - usefull for following particle rotations
    • Axis - also usefull for following particle rotations
  • Special modes (not rendered as halos)
    • Line - a line of user definable length is created in direction of particle velocity
      • creates strand texture coordinates along every line
      • rendered as a thin view aligned quad face
    • Path - shows the whole path of a particle
      • creates strand texture coordinates along every path
      • rendered as a thin strip of quad faces
      • different resolutions for view/render
      • experimental adaptive rendering
        • user selectable thresholds for curvature and rendered segment length
        • allows not creating render data for path segments that don't contribute to the rendered result -> faster rendering
    • Object - duplicates an object in the place of every particle
    • Group - duplicates the objects in a group sequentially in the place of every particle
    • Billboard - creates camera aligned squares in place of particles
      • creates uv-coordinates for every face
      • uv-split option allows using a grid of images to have different images on different particles
      • billboards can be aligned with the camera (billboards rotate with camera rotation) or the global Z-axis (billboards have an "up direction")

Children

A totally new concept is to be able to create more (child) particles based on a smaller amount of actually calculated (parent) particles

• actual particles which paths can be modified, softbodied etc. can be used to create a lot of child particles around the parent
• parents can tell children what to do on the path
  • clumping (children are closer to the parent in the beginning/end of their path)
  • kink (different types of rotation around the parent)
  • rough (randomization of path)
    • rough1 is based on children location so it varies the paths in a similar way when the children are near
    • rough2 is based on a random vector so it's not the same for nearby children
      • a threshold can be specified to apply this to only a part of children
      • usefull for creating a few stray children that wont do what others do
    • rough end randomizes path ends (a bit like random negative clumping)
  • some of these path modifiers can't be applied for unbaked particles as the whole path information is needed for the effect to work
• child particles can also be created between the parent particles on the faces or edges of a mesh (edges should work with curves)
  • nicer for fur creation as a more even coverage on the surface can be acchieved
  • some amount of these children can be made virtual parents that act as parents for the particles near them instead of the actual parent particles that were created from vertices
  • care must be taken to ensure enough parents are available because if some area of a surface is not covered well by parents the children won't nescessarily know what to do and unwanted child distributions can occur

Variables in time & space

Many properties of particles can be varied in time or by emission location with:

• Ipos, which are divided in to two groups
  • Emit ipos
    • change initial value depending on when they are emitted
  • Particle ipos
    • change value depending on current time relative to particle lifetime or absolute current time
• Textures
  • change time depending on where they are emitted
  • can use ORCO or UV coordinates
  • finer control than with vertex groups
• Vertex Groups
  • change time depending on where they are emitted

What can effect what (more can be added)

	emit ipo	particle ipo	texture	vertex group
emission time	x		x
emission location			x	x
lifetime	x		x
velocity	x	x	x	x
angular velocity	x	x	x
size	x	x	x	x
length		x	x	x
gravity		x
brownian force		x
damping		x
drag		x
clump		x		x
kink amplitude		x
kink frequency		x		x
kink shape		x
rough				x
billboard tilt		x

In addition to these textures and material ipos can be used to change color and other render variables. Material ipos also can work in relative or absolute time.

Dynamics

Nothing

• no dynamics done
• particles are born & they die but they allways stick to the emitter

Newtonian

• basically we just add all 'forces' affecting the particle and calculate new loc, vel & rot
• 'forces' can be:
  • texture force
  • physically based effectors (F(X,V))
  • drag (velocity & size dependent air drag)
  • brownian (random force)
  • non physical effectors (anything can affect anything, F(X,V,F) X(X,V,F) V(X,V,F))
  • global acceleration (i.e. gravitation)
  • damp (velocity multiplied by something < 1.0)
• finally we check deflection and see if that changes anything for the particle

Keyed

• particle paths are determined from emitter to another particle system's particles
• allows creation of chains of systems with keyed physics to create long strands or groovy moving particles
• basically particles have no dynamics but are interpolated from one system to the next at drawtime

Boids

• behaviour
  • boid brains can handle only a certain amount of information at a single time
  • the rules are gone through from top to bottom (order can be changed) and once the boid's information capacity is full the rest of rules don't contribute to the movement
  • the rule weights can be from -1 to 2
    • normal behaviour can be expected with weights between 0 to 1
    • from 1 to 2 the boids over react to the rules
    • from -1 to 0 the boid reacts contrary to the rule
• physics
  • boids have a certain maximum velocity
  • average velocity is the usual velocity the boid would like to have
  • the acceleration factors determine turning(lateral) and forward(tangential) accelerations relative to the maximum velocity
  • 2d
    • boids are constrained to a surface of an object (if specified) or to a certain z value
    • if there are no faces "below" the boids xy coordinates the default z value is used
    • the "level" factor has no effect with 2d boids
• boids try to avoid deflector objects (works best for convex surfaces, some work needed for concave surfaces)
• spherical field effectors can be predators (negative strength) that boids avoid or goals (positive strength) that boids try to get to

Structures

Here are the basic structures the new particle system has:

ParticleSystem

• stores particle settings, current particles and baked particles etc.
• basically a container for all particles stuff

ParticleSettings

• stores object independent data for a particle system (like totpart, type, where to emit from)
• is a datablock so can be shared between objects

ParticleData

• currently named so to not overwrite old Particle struct
• holds all single particles individual non dynamic properties

ParticleKey

• stores just location, velocity, rotation, key time, softbody weight and a flag
• when system is baked an array of ParticleKeys is filled with the baking data

ParticleEditPath

• holds all data needed to edit baked particles in the particle edit mode
• not saved in files

System types

Each object (currently only mesh but should be easy to extend to other object types later) can have many particle systems which can be one of the following types:

Emitter

• the usual particles
• are emitter from predetermined places at predetermined times
• use normal newtonian physics or boids

Reactor

• emit particles when a certain event happens
• for example emits particles when a particle from different p.sys either collides with the reactor object or dies naturally
• enables fireworks etc. like current "multi"-feature
• can also "store" collided particles to make sticky collisions
• emitted particles use normal physics
• could be extended to do other cool stuff like deform object on particle collisions etc.

Effector Particles

• particles can be made to behave as effectors to other particles
• allows very interesting dynamics
• activated by the same kind of "PSYS" switch in the "fields & deflection" panel like softbody
• one special new effector type is harmonic
  • normally other particle effector types' particles affect all target particles, but harmonic assigns a single particle to a target particle that acts as a time dependent dampened harmonic oscillator zero point
  • allows creating positional keys for particles with disabling effector particle dynamics, in this case the initial velocity of the effector particles is transfered to the target particles when the effector particles die

Flow diagrams of main particle code

http://users.utu.fi/jhkarh/media/code/particles/particles_main_code.jpg http://users.utu.fi/jhkarh/media/code/particles/particles_dynamics_code.jpg http://users.utu.fi/jhkarh/media/code/particles/particles_cache_code.jpg

Current progress

17.6.07

• bugs fixed
  • using particle modifier activated in edit mode crashed
  • particle mode crashed without path draw mode when animated
  • quiting Blender while in particle mode crashed
  • converting an emitter curve etc. to a mesh crashed
  • allmost anything that applied modifiers permanently crashed so now applying modifiers removes particles too, since the particle modifier is no longer in the stack
• new things
  • particle mode specials menu now has option to subdivide keys in particlekey select mode
  • CTRL-SHIFT-TAB toggles particle mode (just let me know if it causes some trouble and I'll take it away)
  • proportional editing hotkeys work in particle mode
  • emission from curves & surfaces etc. should work a little better
  • new particle distribution type "grid"

Last progress report here, as from now on the changes can be seen in the branch commit log. 28.5.2007

• bugs fixed
  • deleting all particles in particle mode crashed
  • billboards crashed without an active camera
  • adding particle ipos from the "ADD NEW" menu crashed
  • duplicating objects that had edited particles left wrong pointers leading possibly to memory leaks
  • billboard tilt drawing didn't work in relative time
  • using uv-coordinates in a texture without actually having any uv's crashed when rendered
• new things
  • texture field effector with the old rgb & gradient options + new curl option
  • option to show child particles in particle mode
  • billboards support "tiled animated" textures
  • separated explode functionality to a separate modifier that uses the previous particlesystem in the stack
    • due to this old files with explode being used may not work correctly (not guaranteed to work but to try circumvent this use an old patch and disable the explode, then open with new patch and add a new explode modifier below the particle system)
  • weight painting of particle attributes updates interactively
  • curve guides work interactively
  • particle instance modifier has options for "normal" instancing / "along path" instancing and options for unborn, alive & dead particles
  • curve, surface & text objects can now emit particles
    • for now the modifier stack data is not used for these types
    • still very new code so please test to death!!
  • random rotation factor for billboards
  • slightly better baking (less memory allocations) so if you get trouble with baking it's probably a new bug
  • very early experimental "branching" feature for child paths.. for lightning etc.
    • unfortunately the halo vector rendering is not quite up to par to render these lines properly, but like I said it's still very early
  • lattice support
    • the first lattice in the modifier stack after a particle system is used for the particles of that system
    • separate option to apply lattices to baked particles

Leaving for a week in Bulgaria so I thought to leave you guys something to play with. So in other words you have a week to find me some pretty darn good bugs! :)

---

17.5.2007

• started using svn
• code cleanup
• more comments added to code
• bugs fixed
  • child particles were calculated wrong sometimes
  • changing particle systems from the effects panel while in particle edit mode crashed
  • baking reactor particles crashed sometimes
  • react to near didn't work for baked target particles
• new things
  • renamed "exist" vgroup to "density" which is probably more descriptive
  • looping particle lives (baking not yet working properly)
  • instance modifier can instance to child particles too and use the object's track axis for orientation
  • new "flat" factor for child particles from particles
  • halo-line rendering works for path & line visualizations
  • vortex effector works more like with the old particles
  • removed "rotator" effector as there wasn't really any need for it
  • strand widths can now be set in actual Blender units (in material settings) with minimum fallback width of 1px
  • strands can be fully textured by overriding a set of uv coordinates which are mapped to the whole strand
  • textures can now affect both material and particle attributes properly (new button "PAttr" in material map to buttons)
  • edit mode w-key menu now has "select roots/tips" option when in "select point mode"
  • in edit mode "ctrl-+/-" works like select more/less

Phew, it was a long break from coding! Now that my summer break has started I can hopefully put much more time into particles. Happy bug hunting as usual! :)

If you have found a bug before and I haven't fixed it yet try to reproduce it with this new version of the patch. Some bug reports have been very difficult to reproduce so if you still get the bug try to give me detailed reproduction instructions or mail me a bug blend.

---

25.2.2007

• bugs fixed
  • particle paths with softbody didn't show correctly in edit mode
  • wrong tooltip in rotation interpolation selection
  • when the number of emitter elements (verts/edges/faces) was changed the particles were left with incorrect indices to the elements if they were baked -> crash
  • particles weren't reset allways when bake was freed
  • vertex group negating was missin update call
  • comb brushing particles with "keep lengths" off didn't update new lengths
  • wrong name on softbody weight value in particle mode n-key panel
  • particle mode brushing didn't use user preference for L/R mouse button
• line visualization now has an option "speed" to multiply line length with particle speed
• new kink mode "roll"
• new brushes:
  • "weight" for softbody weight painting
  • "soft" for smoothening strands
• cut brush now uses power value to only cut a percentage of the hair
• max banking/update parameter for boids

The bug hunt is on! as usual..

---

17.2.2007

• bugs fixed
  • rotation kink didn't work
  • wave kink didn't work like it was supposed to
  • keyed particles didn't work with more than 1 "chained" system
  • some time ago multi uv's didn't seem to work but now all tests went fine (so not really a bug fix but nice to notice)
  • axis visualization had wrong axis colors
  • particle rotation had wrong axis handling
  • transform code did weird things to particle keys in edit mode (some issues with transform center still)
• particle mode
  • cutting a hair to the root now removes the particle
  • "add" brush with option to interpolate from existing strands (for now works only for particles from faces)
  • separate size,power values for different brushes
  • got rid of opengl select usage
  • edit mode now works in bake space so saving in particle mode keeps the space right
  • deleting keys/particles with x-key
  • moved rekey to specials (w-key) menu
  • transform code handles object & geometry space correctly
• some nonzero default values added for boids
• geometry space works now for particles from edges & verts too
• modified children curl calculation to allow "non helical" curls by using the amplitude & axis parameters
• split particlesystem into 3 files & major warnings cleanup (some strange bugs might come out of this if I forgot to add some includes into the new files)
• option to get baked rotation from keys, direction or incremental direction (works properly only with path visualization)
• particle instance modifier (very much a wip)
• "braid" kink mode
• particle strands now have default softbody goal weights from 1 to 0 from root to tip, these are used if softbody "goal" is setting is set and can be edited in particle mode ( roots allways have a goal weight of 1 even if the goal setting is not used). Loading of previous baked particles may be corrupted due to this.
• baking into world space no longer shows the baking process as it's much faster this way.
• all baking now shows framecounter as cursor, in other than world space baking the cursor goes from 0 to 100 as the baking does not happen in actual time
• entering particle mode the first time disables autobake untill bake is freed
• dupliobjects now take the objects track & up axis into account
• particle visualization is now drawn on top of paths

This time I tried to focus on improving usability, so hopefully a bit nicer testing with this patch!

---

5.2.2007

• bug fixes
  • harmonic effector was missing damp factor in ui
  • one keyed particles bug
  • changing explode "use sizes" now updates calculation too
• recoded part of reactors, they should work much better now
• just a note: Any modifier that's before particles must have same render & view settings, for example subsurf with different render & view level will cause strange things when rendering! So don't come crying to me saying I didn't warn you :P
• baking terminology change: dynamic->world, static->object, animated static -> garbage bin :)
• new baking space: "geometry" allows for baked hair to stick to fully armature (or otherwise) deformed meshes with proper rotation handling etc. (this is way faster & better than animated statics so it was removed) http://users.utu.fi/jhkarh/media/code/furry1.avi
• n-nearest search for kd tree
• far better children between thanks to kd-tree improvement
• some ui improvement
• boids physics type http://youtube.com/watch?v=GUkjC-69vaw

New patch time! Find me some good bugs and I'll try to work on more particle mode brushes for next time :)

---

Progress report list was getting a bit long so old progress is now available in this text file http://users.utu.fi/jhkarh/media/code/particles/old_progress.txt.

---

Jhk 00:03, 20 June 2007 (CEST)