Dev:2.5/Source/Installation/ResourceFilePaths

Resource File Paths

At winter camp, we agreed that we should move to better folder locations for resource files, especially on OSX. For example, moving the contents of the .blender folder that's hidden inside the application bundle to /Users/UserName/Application Data/blender/ where it's easily accessible, and as the platform standards dictate.

We should apply the same logic to the .B.blend file. For such an important file, it's terribly inaccessible and difficult to edit. It's necessary to be able to access this file for various reasons including:

• For testing/debugging, being able to rename your existing config and test by making a clean file, and renaming back after testing
  
• As an artist, packing up your config to take to a new workstation
  
• Switching between configs for different users on the same workstation

This is currently very difficult:

• On Mac OS X, files beginning with . are hidden by the finder, so even if they were to by chance look in the non-standard location where it's kept, most users wouldn't even know it exists.
  
• On Windows, dot files are visible in Explorer, but difficult to manage. It's impossible to rename a file to .something.ext, it requires characters before the first dot.
  
• Even on many linux systems, dot files are hidden by default in the file browser.
  
  

Proposal

A solution involves three simple points:

1. Move the .B.blend inside the config folder (this is already the case on Windows)
2. Un-hide the individial config files and give them sensible names
3. Keep the config folder in the location dictated by the platform standards

Either a clarification or a contrary view:

The goal of this proposal is admirable, but the implementation might need tweaking to get the desired result:

One of the biggest reasons that most platforms prefer that we place user data into per-user profile folders is to support multiple users on the same computer. For this reason we don’t want to literally MOVE the config folder from inside the Blender installation to a place dictated by platform standards. Instead we want to use the platform-specific user area to override the settings kept in the program installation.

On most modern platforms the person (or process) installing software does not have access to other users’ profile areas. Therefore the installation procedure should not put the Blender program in one location and the config files in another area because that area would be that of the currently logged on user. The next person to log on would be missing the config folder. This problem is very big when the installing person is an administrator for a large network.

During installation we need to copy the entire program to one folder tree. However, during execution we need to treat every part of that directory as read-only. The program can read a supplied “.b25.blend” or “startup.blend” for defaults, but this file must not be saved to. Any user settings should be saved in the platform-specific per-user file area, as outlined in the document above. This way the program will operate as expected even if the user config folder does not already exist. We also have the added benefit of being able to easily “revert to defaults” by reading the global prefs delivered with Blender.

- harley

Details

Renaming the config files:

• startup.blend      ( .B.blend )
• bookmarks         ( .Bfs )
• file-history           ( .Blog )
• bpython/menus   ( Bpymenus )
• languages         ( .Blanguages )
  

Config folder paths

Note: At least on Windows, there are proper API calls to get the correct config folder, accounting for localisation settings etc. In Unix, use getenv.

Windows:

C:\Documents and Settings\Username\Application Data\Blender Foundation\ (XP) 

C:\Users\Username\Application data\Blender Foundation\ (Vista)

Mac OS X:

/Users/Username/Application Support/Blender/

Linux/Unix:

${HOME}/.blender/

All these will be unified and replaced by EnvironmentVariables.

Some feedback and further thoughts: [Elubie]

Ok, first some good ideas and some that I think can be further refined here on the page. So I'll for now just dump my thoughts. I like the previous idea of storing the config files in a different directory per version so several versions can coexist - is nice for renderfarms and such I guess.

I think it is important to first summarize about which files we are talking, so I have looked at a blender installation and am making a few comments on where I think the files should go.

File locations

There are I think four types of files:

1. BLENDER APP: Files that are installed with Blender and never changed by the user
2. USER HOME: User specific settings and configuration files (.B25.blend, .Blog, .Bfs)
3. SHARED HOME: Files that can theoretically be shared by the user - at least Windows and Linux provide shared locations for multi-user installs.
4. TEMP FILES/AUTOSAVE: These should go into a folder within the user's home directory so no other user's files are ever used.

BLENDER APP: Is obvious, the blender executable, any locally installed shared libraries by blender, but also things like language settings probably fall into this area. This location is only readable by the user normally. These files should be installed, where the platform usually installs exectutable files (user/bin for example or C:\Program Files\Blender Foundation\Blender). I also see the localisation files and the python lib installed with blender in this group. Basically everything that Blender needs to run should be there.

USER HOME: These should go into the platform specific user directory - the home directory on Unix/Linux or the Users\<username>\AppData\Roaming\Blender Foundation\Blender for example on Windows etc. - and this should conform with platform specific settings. These files are readable and writeable by blender and the user. And these should really only be created by the user - blender has the default .B25.blend built in ('Load Factory Settings') so the Save User Preferences should go to the user specific directory. Same with .Blog, .Bfs and other similar files. Blender doesn't need these files and can create them on the user's request or utomatically. And I'm thinking that even for a multi-user install these should really be per user - for example in a class setting I don't want any student overwriting a .B25.blend for all other students in a shared area). Since these files are not even installed, but created by blender when running, there is no issue with providing a multi-user install executed from and administrative user account.

SHARED HOME: Not 100% sure what should go in this area - maybe shared python scripts etc.

TEMP FILES: I am thinking about something along the autosave folder that we have now within the .blender folder also for temp files. Maybe call it renders or whatever - this is probably the only folder that the user might want to place somewher else.

Other files probably need to be detemined on a case by case basis, but I think with a general rule that everything that blender needs and that is officially released, should go in the installation directory with the blender executable and shouldn't in general be manipulated by the user whereas add-ons and plug-ins etc. are user specific or go into the shared area for several users.

Open issues

Single directory install

We do want to provide a 'single directory' install to keep blender portable. This is done currently by the provided .zip file download, but also by the current installer, if the .blender configuration folder is stored in the application folder where the blender executable is. I think this option needs to be removed from the installer as this has lead to issues. People wanting a single folder install can just as well use the zip file. Or maybe we provide an option to install into the user's 'home' directory as 'single directory install'. It happened to me that for the Blender course I taught the .blender folder was installed next to he blender.exe and the students couldn't save their user preferences because of this.

Scripts installation

The next issue is how to install the python scripts. In the past these have been welcome additions, but there hasn't been a huge distinction between scripts installed by blender and those written by the user. For Blender 2.5 onward, python scripts play a much bigger role in the UI, so missing or inaccessible scripts will make blender behave really badly as well as outdated scripts from previous installations. However, we still want to give the user some possibility to customise or adapt and improve the scripts.

My basic idea to solve this would be to define a user scripts directory where the same directory structure will be kept as for the installed scripts. For mutli-user installations the same folders will be created in the shared files area ('Default User' on Windows, 'usr/share/config/blender' or so on Linux maybe). Python will then look for scripts and register them in the following order: USER HOME ==> SHARED HOME ==> BLENDER APP

This way, the user can copy the blender scripts into his home directory (could also be optional installer setting), change them and still be able to just delete them and fall back to the default provided scripts if things get messed up. Also for bug reports it would be easier to verify by just asking the user to temporarily rename his scripts folder and verify with the maintained scripts provided by the installer. The shared home could then be used for teachers or in studios to provide commonly used scripts.

quit.blend naming

There is an issue when more than one instance of blender runs simultaneously and the second instance overwrites the quit.blend of the first one when closing both apps. One possible solution would be to have a recovery folder within the .blender configuration folder where each process would write something like <date_and_time>_<process_id>_quit.blend For practical purposes I think this should be enough to avoid these conflicts and blender could show the present files and ask the user to choose which one to recover. Instead of nagging the user if he wants to open the last quit.blend, a better solution would be to provide a menu entry to just delete all recovery quit.blend files. (Any better solution to this is of course welcome :) )

More Notes [harley]

Elubie's comments almost exactly echo my own thoughts on this issue. First I'll clarify, then add some more comments.

The zip file allows a user to put the multiple versions of Blender anywhere they want, while the installer just copies the files to the usual system application folder. The installer should do nothing extra beside adding application shortcuts and perhaps adding file associations. It should not create any files or folders in the USER HOME folder.

Blender would not attempt to save any files to the BLENDER APP folder. It would instead save user data to the USER HOME folder when needed, but would not require that any files or folders exist in the USER HOME folder first.

The above does not hamper a single user on a single computer, but does allow multiple users to share the same computer, and also allows multiple users to use multiple computers as found on large installations. It would also allow non-administrators to run Blender on all platforms.

Comments:

USER HOME is referring to a folder that is meant to hold per-user application file data. On Windows this is normally in an "Application Data" folder in the root of the user's profile, while on Mac this is a similar folder called "Application Support". On Linux this would normally be a hidden directory in the user's folder (".blender").

The use of these folders is identical to how Firefox stores user profiles, for example. However, by design each of these locations is meant to be one that users can't get to easily. This is not a location that a user chooses to store a document, like a project's blend file. So we'd also need to add another folder type we can refer to as USER DOCUMENTS. On Windows this is the "My Documents" folder in the user's profile. On Mac and Linux this is just the root of their user profile ("~/"). This is the default "save as" location for blend files and renders.

I would prefer if we can think of some way of not needing SHARED HOME. Locations like the "All Users" profile is meant as a place to put machine-global shortcuts on the start menu and desktop, but not for shared files. In most cases the folder is read-only for non-administrative users. In almost all large installations we want to avoid areas where multiple non-admin users can both read and write, as this becomes a virus vector. Most importantly, anything in a SHARED HOME is only shared between users of the same computer and doesn't do much for large installations: administrators don't want to install the same things on every computer, if that can avoided. Let's see what might need to go into this SHARED HOME first.

As for scripts, my thoughts are that we've probably got lots of reasons now to put all of the delivered scripts into a subfolder of the BLENDER APP folder, and also allow a local "scripts" folder in the USER HOME folder. This would allow any user to create/install/use their own scripts, but would make it difficult for them to delete/wreck the scripts that Blender needs. It is pretty easy to read through the contents of two folders instead of one.

Re the "Renaming the config files" above, I would prefer to add file extensions to bookmarks and file-history. File extensions are not unusual when found on Mac or Linux, but it is unusual when they are missing on Windows. If these files are human-readable then "bookmarks.txt" and "file-history.txt", or if not, "bookmarks.bfs" and "file-history.blog" or perhaps a ".bin" extension?

COMMENT TO ALL ABOVE: I am writing a full doc about the config files and the API that will give access to them as well as other Blender files, where I hope to cover all the points described above. Some quick notes in cases someone wants to look at them: GHOST_getUserDir and GHOST_getSystemDir will provide the location of storage for configuration files, there is only one place for system config (BLENDER APP + SHARED CONFIG, and it is read only) and there is no need to write to user locations while installing (it never passed my mind because in my experience in Unix land it never happens, the config files appear after first launch, and sometimes not even then... which is what I am trying to do). The other part of the topic is in BLI_bfile.[hc], it maps common C file ops and supports the special case of config files, among other things. Stay tuned. Gsrb3d 23:22, 31 March 2010 (UTC)