As I previously pointed out, there’s a huge need to have a clear distinction between the user preferences and the user data. This is already covered by the FreeDesktop XDG folders specification.
Amongst the advantages of following this specification, I list :
- a lot less cluttered $HOME (my mother will not cry anymore when the gtk file selector randomly choose to display hidden folders)
- Make backups a lot more safer and easier. (you know that backuping your $XDG_DATA_HOME along with your files is enough)
- A lot easier to reset a default configuration if you want/need it (and without any risk to loose informations). Even for the software itself could choose to reset $XDG_CONFIG_HOME if needed.
- Avoid some strange bugs that happens because you had a old version of some configuration file
- A lot more of flexibility and portability because no path are hardcoded. You use the XDG library that does the job for you. If you don’t want the dependency, implementing the XDG specification is only a few lines of code.
How does it work ?
Your application should not have its own folder anymore (and should not use another software hidden folder like .gnome2).
User data should go into $XDG_DATA_HOME (which default to .local/share), user preferences should go into $XDG_CONFIG_HOME (which default to .config) and cached data should go to $XDG_CACHE_HOME (which default to .cache).
Of course, there’s no need to read the environment variable yourself : most language provide a library to use XDG folder. For example a patch for a GTK application or one for a python application.
But how do you know what goes in what folder ?
There’s a little trick : just imagine that it was deleted. What would you think as a user ?
If you would cry, running frenetically to your backup, it means that it belongs to XDG_DATA_HOME.
If you would think “Damn, I will have to reconfigure all”, it belongs to $XDG_CONFIG_HOME. This includes user installed plugins even if those plugins might themselves have files in XDG_DATA_HOME. The exception I see is the usernames/passwords for external services. Those are clearly configuration stuffs but, imho, just fit better in XDG_DATA_HOME.
If you would just think “It’s bloody slow those days” then it is obviously part of XDG_CACHE_HOME.
Of course, I’m sure there is some corner cases. But take the time to choose carefully instead of just putting everything in XDG_DATA_HOME, defeating the purpose of this specification.
Think about your users !
A lot of applications are already following this specification (VLC, Wormux, Totem, Metacity, Cheese, Deskbar, Pyroom, Getting Things Gnome) or will soon (Rhythmbox has it in SVN, GStreamer will have it for 0.11,..)
GNOME projects are listed in the tracking bug and you can help to complete the Gnome goal proposition.
And you ? Is your application already XDG compliant ?
PS : If you are not the maintener of any applications, sending a patch to your favourite project might be a great contribution. If you never contributed code to any project, take this opportunity : writing a XDG patch is really easy to do and it could be your first step to world domination.
The Modify your application to use XDG folders by Lionel Dricot, unless otherwise expressly stated, is licensed under a Creative Commons Attribution 2.0 Belgium License.
i don’t see why GStreamer can’t do that now.
other apps manage to that without talking of "migration strategy" and so on (Totem, comix… (1) )
I don’t even know when will be a 0.11 version of gstreamer, maybe in 2012 ?
Here are some apps that comply to the xdg spec (listed from my .config dir) :
autostart
enchant
midori
takenote
brasero
gtg
monitors.xml
totem
comix
gtk-2.0
parcellite
transmission
compiz
menus
specto
Trolltech.conf
vlc
here are some apps that are still not compliant (from my /home dir)
.fricorder
.kinorc
.gconf
.spumux
.adobe
.gconfd
.macromedia
.ssh
.avidemux
.gegl-0.0
.subversion
.bash_history
.gimp-2.6
.sudo_as_admin_successful
.gksu.lock
.mozilla
.synaptic
.bzr.log
.mplayer
.gnome2_private
.gnupg
.nautilus
.themes
.dbus
.gphoto
.openoffice.org
.thumbnails
.devede
.gstreamer-0.10
.update-manager-core
.dmrc
.gtk-bookmarks
.update-notifier
.gvfs
.usb-creator.log
.dvdcss
.ICEauthority
.pulse
.esd_auth
.icons
.pulse-cookie
.evolution
.purple
.Xauthority
.java
.quodlibet
.xchat2
.fontconfig
.jbrout
.recently-used
.xmltv
.fonts.conf
.kino-history
.recently-used.xbel
.xsession-errors
If all apps could comply, an app like Deja Dup could allow to easy backup your preferences
(1)
NEWS: totem-2.21.94:
Note that configuration files are now in ~/.config/totem/ instead
of ~/.gnome2/Totem. You will need to move the files by hand
Comix 4.0.3 – 2009-02-22
$XDG_CONFIG_HOME and $XDG_DATA_HOME directories instead of in ~/.comix/.
When installing, comix copy the files it needs from the old directory and, at 1st launch, it asks the user to confirme the deletion of the old files unless the user need to go back ton an old version of comix.
How are you supposed to find the values of those variables? On Ubuntu 8.10:
$ echo $XDG_DATA_HOME
$
and same for the other ones. Maybe ‘xdg-user-dir’ is supposed to help, but:
$ man xdg-user-dir
No manual entry for xdg-user-dir
The python patch you provide doesn’t show anything (useful). How to retrieve xdg_data_home etc. is not shown. I shouldn’t ask but… did you read the patch before posting ?
Stefano, the key line is:
from xdg.BaseDirectory import *
Try a "$locate xdg/BaseDirectory"; it is a very simple Python module and easy to understand.
I assume that by "folder", you really mean "directory". A "folder" is something to put bits of paper into. The word has no meaning in a software context.
More importantly though, I have mixed feelings about this scheme. While I agree that it puts configuration files conveniently together, there is something to say for the fact that $HOME is also convenient. Configuration files tend to start with a "." too, which makes ls(1) not list them unless you use "-a".
(somehow my previous comment got sent before I could finish typing it).
… basically, I wonder what "problem" this scheme is trying to "solve"? I can’t see anything particularly wrong with the way dotfiles have worked for the past thirty years. Why does it need to change? You post mentions a number of bugs in pointy-clicky applications and backup schemes. Wouldn’t it make more sense to fix the pointy-clicky applications and the backup schemes than to break with decades of established tradition?
Robin > If the variable is not defined, then the default value should be used (see the spec).
Stefano > I wrote that patch
And it doesn’t read environment variables because I conveniently use the XDG library.
Philip > You didn’t read the post, didn’t you ? The problem is the mix between "data" (something you want to keep) and "preferences" (something convenient but you can afford to lose it or, sometimes, you just want to reset it to default).
The fact that I had the problem and that I discovered that a specification already cover this problem should outline the fact that the problem exists. Everybody might not be affected. Old timers might even have developed a inconscious strategy to workaround that problem. But the problem still exist. One simple example : I want to backup my /home to transfer it on another computer. What should I take ? I cannot take the full home because there’s too much .hidden crap and because in those .hidden, some might contain preferences that are very specific to this computer (just one example, the .alsa or .ICE stuffs). That’s a problem and this spec solve it in a very nice way.
A nice side effect is that you can easily debug application with some dev-only data :
bazaar.launchpad.net/%7Eg…
In fact, the only criticism I can see about this spec is "Grrr, I need to change my habits".
Yes, such a patch is easy. The difficulty comes from doing the migration nicely.
Argghhhh! Didn’t see that line ! Thank you !
Colin > I agree. But such a migration occurs also when you want to change your format, it’s not really different. For a silent migration, see Rhythmbox in svn. For a migration that warn the user, see Comix.
Also, in some simple case, you can just dismiss the data (like in gweled case where only the size of the window was saved)
<quote>The exception I see is the usernames/passwords for external services.</quote>
Shouldn’t those be handled by gnome-keyring or some password manager rather than stored in a file?
Alexandre > ideally, for the passward, yes. But the account login should be stored somewhere. Evolution should store somewhere that your email accounts are blabla@homemail and bla.bla@workmail. I think it belongs to XDG_DATA_HOME, even if it could be seen as configuration.
ploum: "preferences" are data too. You spend *years* writing a good muttrc, so of course it goes into your backup too. Never mind private keys and the public keyring of GNUPG or other such data. That all just belongs in your home directory. And you simply back up your entire /home volume.
In the Unix environment, settings specific to a single machine are not a problem at all. I share my home directory over NFS between BSD, Linux and Solaris without problems. Applications can easily be convinced to use files specific to them ($HOME/.foo.$HOSTNAME, or $HOME/bin.SunOS, for instance). There is no need to jump through hoops.
Despite reading the post again and all the comments, this feels like it’s solving an imaginary problem. Those who do not understand Unix are doomed to reinvent it badly.
If applications cannot be told to use files specific to a machine or architecture, they need to be fixed. If your backup policy does not take a backup of your complete /home (or /exports/home or whatever the hype is this week), it is broken. If applications break when you restore your backups, they need to be fixed.
I have been using Unix of some description for literally most of my life and I can’t imagine having any of the problems you are describing. I take complete backups (and I *know* they work because I often fry hardware) and I very often use many different machines running on different architectures. It Just Works[tm].
I think the problem you are trying to fix is lack of user clue. The solution is to be found in education (and quite possibly a hefty clue-by-four), not in technical hackarounds which will only come and bite you in the ass eventually.
Repeat after me: keep it simple. Work with the system, not against it. Unix can be your friend if you take some time to learn the whys in addition to the what-the-fucks.
Oh – and of course you store Important[tm] configurations under version control and systems like that get a bit upset if suddenly all files become symlinks — like I recently had to do for Awesome 3 whose configuration moved to "$HOME/.config/awesome/rc.lua" instead of being in "$HOME/.awesomerc" where it made perfect sense.
A quick grep shows that all but fourteen of the "."-files in my $HOME are under version control and some have been for nearly a decade. Moving them to a directory and symlinking them is just a pain in the butt and it doesn’t solve any problem. There is no problem to solve.
Three of the "."-files which are not under version control are sockets and most the others are things like .Xauthority and friends which are easily regenerated if they are not right…
Philip > I will never pretend that I know Unix because I’m nothing compared to you. Yes, you rock.
But maybe there’s something I’m understanding better than you do : end user. Like my mom and Joe the plumber. And they have a different approach, they have different problems. They don’t have a muttrc, they cannot even use a console and don’t talk to them about "education", they just don’t want to learn, they want to use.
With this kind of users, a new set of problems appeared, a new set of softwares (Evolution instead of Mutt), and thus, another set of problems. I just started to wrote an example then I saw it was becoming too long as a comment but the problem is real.
You told us about you muttrc being data but that should tell you how far from the end user you are. You are considering your "software configuration" as an important piece of work ! Most people in the street don’t even understand the word "configuration".
And I believe that the computer should adapt itself to those people but you might disagree.
(Thanks for the comment, I like being challenged
)
Are you aware of any effort by the KDE crew? Did you get in touch with any of them?
Diego > I don’t have any contact with KDE developers. I think that some KDE apps already follow the spec but I hope that someone will do the KDE evangelisation
"end user. Like my mom and Joe the plumber"
Mom agree with you
Philip > So the fact that they are sockets and auto-generated file like .Xauthority mixed with your config files doesn’t botter you at all?
Hi Ploum,
The XDG spec is a good solution, but it doesn’t solve the whole problem. I explain why hereunder.
Take an example of a home directory today :
$HOME
|-.nautilus/
|-.mozilla-thunderbird/
|- …
|- A joke (by your dear uncle).odp
|- Photos/
|- .bashrc
An example of the same directory with the XDG completed :
$HOME
|-.local/
|-.config/
|- A joke (by your dear uncle).odp
|- Photos/
And what must really be done :
$HOME
|-.local/
|-.config/
|- Data/
|- A joke (by your dear uncle).odp
|- Photos/
According to me you get four types of files :
1 – settings,
2 – important data produced by a software but that regular user don’t know,
3 – cache data,
4 – data stored by the regular user.
The key is to not melt the type 4 with 1, 2 and 3. The XDG spec still melt those data, a bit less but there are still in the same $HOME folder.
The true soluce is that $HOME should contain 1, 2 and 3 and not be accessible easily to the regular user. For him, its home is its data folder (4) and all applications have to open it by default instead of $HOME.
I don’t know if I’m clear enough, but to summarize : when I click on my home folder, I want to see MY personaly produced data and only them.
Great post on a great blog. You’re a model of how I aim to contribute to open source community now that I have free time.
@PomCompot: There are both DATA, CACHE and CONFIG specs, so you just have to change the $XDG_DATA_HOME to fix this.
Is there perhaps some kind of website where the XDG status of all applications is listed. (e.g. pidgin – wontfix bug #32323, rythmbox – yes, firefox – bug #3333)?
Cristi: Gnome (applications/modules) has (have) a GnomeGoal for this…
Mats Taraldsvik: I have already seen the GNOME page, but I have been unable to find a page which would have the equivalent for other open-source projects. For example, during my research, I found that Pidgin and mplayer refuse to adopt the XDG standard. I would be really cool if there would be a wiki-like page where I could post this, so that other people would not have to waste their time, and focus on the important thing: advocacy.
Personally, I am 100% FOR the idea of every application conforming to the XDG standard for the same reason that another commenter used to argue AGAINST it. Configuration files in a revision control system.
I have been doing something similar to the XDG standard for a long while now so that it is easy to store large chunks of my configuration files in categorized GIT repositories, but the annoying thing about doing it this way is that it requires me to create a mess of .symlinks in my home folder linking to the actual configuration files/directories. Also, a few applications choke on symlinks and therefore cannot as easily be kept in GIT without a bit of trickery and scripting. It would be MUCH simpler and MUCH less messy if all the configuration files were in a single folder like the XDG standard proposes.
Anyhow, thank you to Ploum for posting this, and for advocating the use of the XDG directories.
I think system related part of GNOME should store settings in ~/.gnome2 or ~/.gnome2_private . GNOME-KEYRING, nm-applet, etc. should store settings/passwords in ~/.gnome2 or ~/.gnome2_private . Pidgin, Wesnoth(especially it), Nautilus, Metacity, Firefox, Evolution should behaves as is described in xdg specification.
Imagine Firefox got broken. I can remove only configuration files, but the data(like bookmarks, skins, installed plugins) aren’t removed. Great think! Problem is to do that with huge application allowing to install addons. In fact Firefox as now allows to store data in database system. Maybe It’s not too difficult?
Finally GStreamer 1.0 should be ready in between October 2009 and March 2010 for Gnome version 3.0
http://www.linux-magazine.com/On...
I wish they don’t forget to switch to XDG folders spec as promise
libxdgmm
I think the Portland Project from freedesktop.org, is a great idea and everyone should be supporting it in their applications. I’ve just created a very small C++ wrapper (libxdgmm) for accessing XDG more easily. To use it, you need libxdg.h and …
With a brand new installation of Ubuntu Karmic alpha 5, there is still some bad boys :
/.dbus
/.evolution
/.fontconfig
/.gconf
/.gconfd
/.gegl-0.0
/.gimp-2.6
/.gnome2
/.gnome2/accels
/.gnome2/eog
/.gnome2/epiphany
/.gnome2/evince
/.gnome2/file-roller
/.gnome2/gedit
/.gnome2/keyrings
/.gnome2/nautilus-scripts
/.gnome2/panel2.d
/.gnome2/backgrounds.xml
/.gnome2/yelp
//.gnome2/accels
/.gnome2/eog
/.gnome2/epiphany
/.gnome2/evince
/.gnome2/file-roller
/.gnome2/gedit
/.gnome2/keyrings
/.gnome2/nautilus-scripts
/.gnome2/panel2.d
/.gnome2/backgrounds.xml
/.gnome2/yelp
/.gnome2_private
/.gnupg
/.gstreamer-0.10
/.gvfs
/.mozilla
/.nautilus
/.openoffice.org
/.pulse
/.ssh
/.subversion
/.themes
/.thumbnails
/.update-notifier
/.bzr.log
/.recently-used
Pas toujours facile de faire la différence entre $XDG_DATA_HOME et $XDG_CONFIG_HOME
Par exemple Specto place la liste des observateurs dans $XDG_CONFIG_HOME et Midori place ses marque-pages dans $XDG_CONFIG_HOME alors que je verrai plutôt ça dans $XDG_DATA_HOME
code.google.com/p/specto/…
http://www.twotoasts.de/bugs/ind...
que faudrait il pour que le GNOME Goal soit approuvé ?
car par exemple, pour Epiphany : blogs.gnome.org/racarr/20…
Actually, the xdg spec is crap, and comment #20 touches on the issue. The spec is written to make you think that you can install programs with –prefix=$HOME/.local and –sysconfdir=$HOME/.config. This is supported by the fact that by default the data dir is ~/.local/share and not ~/.local, as if to mirror /usr/share, like –prefix=/usr –sysconfdir=/etc.
If this is indeed the case then normal programs would not want to put data there, same as they don’t put data in /etc or /usr/share. In this case there is no place to write user-specific hidden data — it’s as if there should be a ~/.local/var (say, ~/.local/var/gnupg instead of ~/.gnupg for the keys).
But if this is not the case and programs should put data in those directories, then it doesn’t make sense to only have "share" in ~/.local, it is just ugly. It also doesn’t make sense to separate data and settings, because you would want to back up both at the same time (think gpg/quake settings in ~/.config and keys/quake saved games in ~/.local/share).
The motivation behind the spec is good, but really, there shouldn’t be stuff like this design-by-committee crap. I can’t see any reasonable discussion, only blind following of this self-proclaimed freedesktop authority. As a dev I am against following the spec until someone revises it to make more sense.
salut! seeing as you seem to be deeply involved in xdg, perhaps you might know… is there a recommended place for files like .bashrc, .screenrc, .muttrc, etc? should there go inside of .config (by default) and be named bashrc, screenrc (eg: without the leading dots?) or should they stay in the home directory.
cheers!
_J
Thank you
I’m fixing in the new releases for all my apps
Now they’re more rationale