make him feel welcome hello everybody i guess i will be speaking kind of fast but still if you’re gonna sleep off the lunch i’m not gonna be mad at you so like feel free to do that I’m mukesh lanka and oh and i can be reach in multiple ways on internet i will be speaking today about configuration about configurability of your applications but that’s kind of a tricky topic so maybe let’s talk about cars american cars rider americans love cast don’t they I know I love American costs so let me show you one this is a car that predates Henry Ford’s birth because he was one in 1863 and this is one of the cars that actually made it possible to colonize the area where we are currently so thumbs up the problem with colonization is that they’re in there is an increasing number of people wanting to get on your wagon okay pun not intended so people were producing wagons here on site people were importing wagons from England and the more tricky parts of this way these wagons were obviously the wheels as you can see there are not really that much complicated and they broke they broke quite easily so you had to change them to change a wheel you use a tool which looked like this it was quite a widely used to and very many people actually had that with their wagon the problem was that an increasing number of wagons were imported from England where the not on the wheels were of a different size so the tool couldn’t really be used to change the wheels from English wagons then somebody named Richard Clyburn came with a tool it’s called the English key first invented in 1842 and independently by Edwin bird budding isn’t that the case for most of the greatest inventions that they were independently in invented by somebody else I guess that’s the case yeah that’s the English key also known as the adjustable spanner the screw range and basically I’d say it’s the first example of a tool that is highly configurable but let me just skip that subject for just a second to tell you about Robert and Peter sig if you don’t know the guy you totally should he wrote the Zen of zen and the art of motorcycle maintenance it’s a great book you should totally read it and he said in the book the solutions are all simple after you have arrived at them but they’re simple only when you already know what they are and this is totally get the case with the adjustable spanner because once the design was known it’s totally obvious I could actually explain my four-year-old son how to use it and how its constructed it’s totally easy but before actually it made people are worried it made people who buy multiple ranges so there is a certain amount of genius involved in every invention of the sort so moving along to configuring applications to actually computing science let me talk about the four desirable characteristics on configuration the first one is that configuration should be composable for instance unix tools usually provide different levels of configuration there are program defaults there are operating system defaults there are user defaults environment specifics and even runtime overrides and all these levels of configuration can be used to specify the same options but there are rules on which overrides which which lets you actually manage the changing environment of your operating system by leveraging some of the configuration to the package maintenance

while being able to have your own specific variants of certain configuration options so compatibility one of those configuration should be readable by people right to distinguish when an option is enabled or it’s disabled but not only by people also by programs because configuration should be cheap to pass then you have exchangeability so configuration should be easy to modify store and transfer that includes the ability for a program to write the changes back to the configuration registry whatever that is and later last but not least discoverability which only means that the configuration should be self documenting so we have those four configuration that should be composable readable exchangeable and discoverable how that how does that look in the real world let me just show you a couple of usually used configuration formats and how they adhere to those characteristics let me start with the easiest one like s or easy formats like any but there’s a problem with any it’s not really a standard it’s an informal standard there are lots of variants lots of different file extensions you have Dora needs CFG conf dot CNF whatever and then there’s all the differences between those variants like can case sensitivity how they actually implement blank lines comments duplicate names how they support escape characters stuff like that stuff like that there’s lots of lots of variants but there are good sites to format like this the good side is that it’s highly composable which is sorta implementation-specific but most of the parsers do support composability it’s readable as long as you’re consistent with your configuration it’s readable and it’s exchangeable as long as you keep the encoding sign of your configuration files it’s kindly exchangeable and if an option is specified it’s kind of discoverable I mean most of you can possibly tell what the configuration file on the slide is about another highly known format not as loved as the first is like the custom Apache format which has a great feature because it supports karoke supports includes as well and each directive is enclosed in a single line there’s this directory VAR www hosts which has this strange options and this is one of the cases where I want to show you how even seemingly simple format can be deceiving what’s the difference between those two option lines well the first one actually forbids indexes whereas the second one enables them that are so custom formats yes there is another widely known configuration format which engines uses kind of see ish easier in terms of kuraki because you have only the three carries the HTTP blog the server block and the location blog but they kind of have a number of possible contexts the global context the HTTP contact server contexts it supports even if else it supports nested locations in locations stuff like that so it’s kind of nice then you have Jason this kind of worries me because Doug Crockford when he came up with Jason he didn’t think of it as a configuration format more of our transmission formats so a transfer format whereas an increasing number of tools most mostly nodejs and friends but also sublime text and other pieces of software uses it as configuration which has good sites because it’s simple kind of human readable cross platform supports multiple languages and unicode but Jason doesn’t support includes

doesn’t support cascading and while adjacent encoder may accept and ignore comments it must not output comments so it’s kind of a weak format for configuration if you cannot have comments and then is the new kid on the block which isn’t really a format yet this is a format by tom preston wanna from github which i believe is the reason why why they why there is so much interest in it in its current stage currently tomo is defined by a single by a single read me dot md on github and it still changes quite rapidly as you can see it’s basically Enosh but supports a couple of a couple of data types like integers or arrays bling and even diets curiously and sorta wants to support the nesting as well as you can see in the server’s example which is quite nice but then actually you can see that this is a format that is mostly made like a wish list of the things that Tom considers good to have which at some points conflicts with one another for instance he says that strengths should be encoded in utf-8 then he lets you specify arbitrary white characters which kinds of in my book doesn’t really play along but to be honest I don’t really be I don’t really want to be too hard on the format yet because it it’s it’s like a month old or two months old then again it’s open source I even have like a couple of pull request accepted in it so I’m kind of responsible for it now I yeah but still it’s in a very early shape so I wouldn’t consider it for adoption in your applications especially that most of the Python passes for it use really strange methods to achieve this parsing so really not yet then there are the complex formats like XML XML has this desirable characteristic of being validated so you can use schema validation which is quite nice for a configuration format as well but we are all tired of XML not being really readable by humans for instance the configuration file you you have on the slide is an actual configuration file from soap which you can see uses some peculiar formatting techniques for kind of hiding the unfamiliar ative of XML then you have another format that wasn’t really thought of as a configuration format but rather than a transmission format it’s it was conceived as a data serialization language technically it’s a superset of Jason supports theorizing arbitrary native data structures which Ruby guys especially the rails guys use heavily and sometimes it leads to security problems yeah but it’s also kind of unfamiliar whoever sees a file like that on the first side cannot really tell how would you extend it I mean if it’s already kind of specified you already see options on your file then it’s kind of readable but how would you add a new option that’s not already there this is kind of Fe it’s iffy because like Tom from github said the yamo specifications is 80 pages long so it’s kind of complicated and it has this feature which we like in python is a programming language that it uses wide space to define scopes I would argue that in a configuration format it’s not a virtue but a problem because it makes things fragile for one thing and make the copy and paste ability of configuration kind of a key and fresher so if we already

use significant white space why not use Python it’s composable composable because if you plump enough you can import anything there’s much power in the hands of the user now but the user can get things wrong especially if he is not a Python programmer i would say that there’s too much plumping here and if the user is not a programmer it’s kind of hard to understand for instance the module root part that is on the slide is what every each and every Django application does to kind of specify where those files should be put so if you’re not a programmer you’re gonna have a bad time actually understanding what is going on here so these weren’t really those bad formats there is also a set of awkward formats for instance DSF like the cucumber example on the slide there are really formats that the users came up with themselves so it’s really a specification language dedicated to a particular problem domain on so or so wikipedia says but technically this is something that has trouble with deciding whether it should be a totally new format but it’s all shooted based on an existing one for instance a very often taken approach is to extend or otherwise change the programming language that is used and to pass this format later on like you see on the puppet example on the slide so there is a problem of shifting domains and then if you want to use a DSL you might just as well specify an external file with a schema and it’s all the same then the sig light which I would argue is a pretty neat format configuration but there are strings attached for instance postfix really uses traditional txt files for configuration but when it comes to lists or hashes it supports sequel I’d files as well which is kind of neat because now you are able to write tools that automate your changing of your configuration at runtime which is really nice then you have the example of mongrel to which stores all of its configuration ins in sick light the problem with that is that only programmers do really know how to actually change the configuration and then it’s very bad at discoverability but mongrel to came up with this great notion of ditching editing of psycho light by hand and now provides you with a tool that’s basically used to change each and every switch in your configuration even if you’re using text based our text based configuration format like Apache does and there are tools for it that make some of it easier for users for administrators you windows registry so I mean it’s comfortable right it’s quirky go and stuff but it’s not really exchangeable it’s highly window specific and moving between per program and user specific settings is difficult it’s also a single point of failure it’s readable sorda but doesn’t support comments and then it is the only configuration format that sports the feature of having a market for tools that cleans them up but it’s not the worst format ever there is one that I want you to show you yes the worst for whatever is the one you conceived on your own because you will impose a learning curve on your users they will find an easier to use version of the software and ditch your application so don’t but then your design decisions will be unintuitive for a class of your users and the thing that you’re so proud of that made this configuration form it’s so different this exact feature will bug your users but it’s not our greatest problem right

the greatest problem is that you will fail at parsing because parsing is quite quickly so possibly not by much but you will that’s our certainty I’d say and then you’ll see that people start depending on the quirks of your partner so that’s a lose-lose situation because at that point you cannot fix the bug without screwing your users it gets even worse because configuration written ones has to be supported forever so please select one of those formats I told you about before they may not be perfect but at least they are already field-tested let me tell you a bit about seeking how much configuration should there be in your application in the first place because the configuration format is really like the technical detail but how much configuration do you really want and how should it be bootstrapped so if your user approaches your application on the first day how should it be how should he be confronted with configurability of your application there’s a great example of Dropbox where Dropbox really lets you configure their server side client in a really peculiar way I haven’t seen an example of that in a wild for any other piece of software basically you are w get or curl your binary you unzip it and then you run the dropbox d and all all it says is that please visit a URL to link this instance to this machine to to dropbox right so i would say this is perfect it makes configuration a non-issue for new users these are the four characteristics I spoke about before but let me handle them from another expected we all know the excerpts from zen of Python a beautiful is better than a great supers balan and complex everybody knows it everybody should know it flat is better than a nested surely readability counts but there should be only one of this way to do it so I’d rather not introduce tools which look like this but rather have tools that look like this actually this is a website i want you to all use because the world would be are totally different place if everybody used make everything okay calm changing the subject of it configuration is not data so if you want to describe behavior in your application this is not the same thing as the things that are subject to behavior which post fix shows you quite in a quite good way by separating the configuration in the text files from the hash tables which can be for instance taught in sickle files i would say that each and every one of you probably knows that hard coding is an anti-pattern like hardwiring circuits and stuff but also the reverse is true so source code that is embedded in configuration is also an anti-pattern because you have to consider testing you have to consider debugging when your user actually files the bug in your issue tracker then you’ll have to ask him can you please send your whole configuration configuration along which is actually quite tricky but configuration is also not code so the thing I said the reverse hard coding problem which actually provides us with the whole picture that configuration only characterizes behavior code defines and execute it and data is only the thing that is subject to behavior and should be stored in some sort of database now quickly Django I don’t know whether everybody uses it here so let me practically tell you about some problems with that Django mixes different kinds of settings and then you have approaches how to solve this because not all the problems are over modular basis some of them are even security problems for instance storing passwords and the circular key so the simplest approach is to just import our local module which stores the sensitive part of the data

but it has problems what happens if the settings local module doesn’t exist so we have all sorts of workarounds for that but there are alternative approaches for instance specifying specifying files for different environments and then running with Django settings module but this is kind of awkward as well so you also have approaches like this where we can specify configuration files like we would do in other UNIX tools so that the settings folder looks like this we can specify all sorts of override of all settings and they are executed in the alphabetical order ah then you have general configurations with which I highly recommend which may configuration a non-issue using classes to define different environment this is really cool because you can also use mixings mix-ins with it so for instance having your own library of possible mixes and then only compose them using multiple inheritance to get whatever the configuration looks like in the end yeah the only problem will fold out is that you have to specify two ways of configuring configuring your application then i would say that up on a flat approach like this using any files would be great as well and this is kind of something that you can kind rather simply implement all on your own to having production settings in another file then staging configuration for instance using config passer you can read multiple files then there comes the problem of mixing file-based configuration with common line overrides for that I will only say use config glue there’s also conf conf arc bars but config blue is the nice part the nice implementation but whatever you do please name your config command line arguments on par with the file configuration settings because if you don’t people are gonna get confused that kind of concludes my confirm I presentation i also have a short detour to show of the new version of the config parcel library in python 3 which supports let me just ha which supports dictionary like API so the old ways are switches switched to a new more much more nicer API it’s very configurable so you can fetch configuration from a dictionary you can create sections from a dictionary the OP is like this yeah and it sports even our build out inspired interpolation which is kind of nice it’s highly customizable which is all documented on the on our website what happens here powerpoint don’t do that to me yes it’s misconfigured totally let me just switch the screen then if it let me yeah but these are the basically the last slides I do have so if there are any questions please go ahead