alright we’ll go ahead and get started people will start trickling in that’s totally fine you guys are hopefully all here to talk about security and defense in depth we’re going to be talking about best practices around how you can secure not only your Drupal site but the stack that it runs on as well and some lessons learned along the way so I’m Nick steal ow I’m director of engineering at Pantheon so we run about a hundred thousand Drupal and WordPress sites and have learned along the way a lot of you know we’ve seen a lot of different attacks and different exploits and learned a lot along the way but looking a platform hoping to share some that today with you I’m Chris title the founder of cellar door media we’re a development and consultancy firm based out of Seattle I’ve done everything from small sites all the way up to architect at large scale ecommerce for actually african airlines and so when you deal with security on the level of major e-commerce it’s it’s a whole new game so yeah and I’m Allie probasco I am the GM of Drupal business up at towns and security we’re a data security company have enterprise clients worldwide and are really excited to be working within the Drupal community so yeah I’m excited to be presenting with two great co-presenters we each kind of bring a different perspective on the on the security world nice perspective is more on a little bit less kind of specific to Drupal but more kind of on the technology on the platform side and so that’s kind of where my expertise will help guide us today and I’m coming at it from the Drupal architecture side from the developer side if you’re a dev shop if you own a shop if you’re a freelancer what can you do to protect yourself and then I come from basically a security company you know we we work with a lot of clients so I understand their you know concerns a lot of people come to us for a couple different reasons primarily meeting compliance or needing to manage their risk of a data breach and I love this slide because it is very true there are only two types of companies those that have have been hacked and those that will be even that is merging into one category those that have been hacked and will be again we are seeing that it just shows that it’s absolutely important to have security at the forefront of your Drupal projects if duple is going to be considered an enterprise-class CMS businesses care about is going to be meet my business needs and is it going to be secure so you have to build security in from the ground up in order to prove that Drupal is the right CMS for the enterprise and does it keep your CEO out of the headlines yeah yeah so I think we’re throughout the talk will be kind of talking about from a couple different angles one is kind of about what you can do to secure your sites and or you know secure your business but then part of it also is just kind of what a security mean to Drupal to the community in general as we want that community to grow son of a breach you cannot afford to be hacked these numbers are real the ponymon Institute puts these out every year these are the latest numbers three and a half million dollars per breach or 145 dollars per record I don’t know too many businesses that could go through one let alone two of those and just an astounding number there’s a website you can go to as a 428 I mean it just blows my mind that there’s been over a hundred million records exposed so times that by 145 dollars per record that’s a lot of money getting spent on data breaches and we’re going to be doing a little audience participation just to get people going before lunch so if you go ahead right now and raise your hand if you don’t want to participate in any of the things we’re gonna do okay that’s looking good I’m very good about this just to start off like who’s gotten a credit card a new credit card sent to them by their bank like I have and actually it just happened that like Sunday evening before I flew down for the for the record the people that are watching this online that was every hand in the room yeah so you know this stuff is really happening you can see some of it with your Drupal sites and the kind of exploits you hear about in the community but you can also you know see it through your daily life through your interactions with your bank or how your online experiences are changing with the different organization to work with online you know and to drive the point home a little bit more this is a public service announcement from the FBI and

from about a month ago and I think they need a little work on their website because it was actually like a lot harder to download this man or just load the page but this is a this is something from the FBI about a month ago I was saying like in the same word basically like Isis and WordPress like like you know like so this stuff is going on right like this isn’t just like oh you know my you know whatever right this is like the FBI and Isis and WordPress all in the same sentence in the past month right so all of that to say you’re going to get hacked unless your site is permanently offline and you’re just using it on your local machine you have to assume that your site is going to get hacked I work with a large company right now and we just did a security audit and they the security team came to us and said plan on your PHP being exploited plan on your database being breached what are you going to do about all of that how can you protect it in the event of everything that you own is being given out so don’t freak out this is all the fun at the top there is fear uncertainty doubt this is what a lot of people will try to push it you and we want to make sure that you guys walk away from today with with practical steps with real-world scenarios that you’re not just running around freaking out like oh my god I’m going to get hacked the world is going to end and I should just stop what I’m doing crawl into a hole so don’t worry well we’ll walk you through it so the first step is building a security consciousness for you and your team yeah so that’s right like you can you know like there are people you can pay to help you with security there are like you know legal entities like from the government that help you handle security there are compliance organizations there’s a whole ecosystem but the one thing that you really cannot buy is educating yourself and so you know when you think about security the thing your responsibility to yourself and your colleagues and the business owners and the sites that you create is to educate yourself and be able to think objectively about security so really it’s about a frame of mind right security isn’t something you implement where you do or you buy or you like you prevent this attack or that attack you know it’s really about a frame of mind I think great security people when they walk into a room are kind of like exits fire escape like no security guard by the door you know are just kind of like noticing stuff noticing how stuff could be broken how stuff could be breached and um you know so you don’t want to be paranoid but you don’t want to be like ignorant either about this stuff well the real goal yeah I also think as Drupal developers you guys have a responsibility to your clients to be setting them up for success and you know if they get a breach or when they get a breach they’re going to say what the hell guys I thought I’d paid you a lot of money to make sure that I didn’t have to go through this and so and I just think it’s you gotta do it yeah from a developer mindset every time I walk into a bidding process or an RFP the first thing I think about is what data is there how do I secure it what do I have to do and that should be on every developers mind as they’re going through this right so you know another way you can look at this is kind of on risk mitigation how much it like how much are you investing security and what risk to those mitigate it’s also important to think of kind of what bucket you fall into if you’re doing a you know a hair salon brochure where website that might not be such a big target if you’re working with a kind of activist or political organization that might be you know could be a target and you’re going to want to invest more so part of it is just understanding kind of you know how in order to understand how much you want to invest in security it’s also understanding what what risk profile you have and whether you’ll get away with kind of best practices or going a little more in depth or a kind of a very lockdown lockdown just a double check and everyone in the back here as yep raise your hand awesome cool just double check in we have loud voices and I don’t like microphone so I’ll tend to walk away from it and come back but hopefully the folks that are going to watch this later online don’t mind that so okay more audience participation who here is working on a project that involves compliance is your hand and I bet there’s probably a lot more hands that aren’t raised because you might not really get that simple things like an email address or a username are also fall under compliance regulations so I bet that that’s going to probably raise a few more hands the thing about compliance it’s not optional and compliance is also the low bar for data security I you know I read a lot of the industry you know magazines and journals and whatnot and and the common theme is that compliance is just the stuff that you should be doing at the absolute bare

minimum it snot going to necessarily prevent you from a breach it’s just like the low bar you’ve got to do more but for a lot of people that can be difficult but I just urge you to just look at a lot of the compliance regulations if you’re just trying to get a feel for what you need to do security wise at PCI DSS is a great one and just look what do these compliance requirements say I need to do and that’s what you should be shooting more right out the gate there’s also some studies i’ll make up some statistics but on that I think it’s like eighty five percent of companies are out of compliance within two weeks of their last audit right so if you’re really thinking about it from a compliance angle you’re probably not really thinking like holistically like but you know you’re kind of thinking about the low bar you really want to be integrating this into the entire life cycle okay so we’ll talk a little bit about the CIA security tried not the governmental CIA stands for confidentially confidentiality integrity and availability and this is not something that we made up so I like the others ya authors opinion you can you can certainly go online and learn a little bit more about this but just to kind of briefly talk about it confidentiality is roughly equivalent to privacy and data encryption is actually a common method for ensuring confidentiality and user like I said user IDs and passwords constitute things that you need to encrypt and also two-factor authentication is becoming the norm and and just one thing I like wanna stress is always err on the side of more confidentiality you don’t want to be the the you know when when the CEO comes down and says what’s going on and you just say away I thought it was secure you know it’s it’s it’s all all these items are a part of a defense and death approach to data security the next is integrity this talks about data integrity is the data that you’re receiving the data that was intended for you to receive are you did somebody man-in-the-middle attack it are you sure that you’re securing your connections in and out and then it also goes back to don’t trust your users just because somebody says that they’re you know that they’re in admin don’t give them the ability to write arbitrary PHP in your notes the admin password could get hacked and just because they say that they’re the password or they’re the admin doesn’t necessarily mean that you should just give them every right in the world to explore your system and then culminating with availability so this gets to kind of denial of service attacks so someone’s attacking your site or your service in a way that it’s not able to provide the value that it’s supposed to to the customer if your site is down that’s not that’s not a secure site and so one of the one of the reasons i think this the CIA tried is important is that like security is a big topic right so it’s really helpful just have this as a very simple framework for kind of breaking down the different types of threats what you know how they might be able to impact your system whether that’s updating data that you don’t know about or getting data that should be private or taking your site down so if you can kind of decomposed security you can start to talk about it a little bit a little more detail the thing i really like about this image is that it’s a full loop and that if you break any one of these your security goes out the door you can have the most confidential system you can have it be one hundred percent uptime and somebody man-in-the-middle attacks you and you just lost everything same thing with availability you can do it all but if you can never access it or the site goes down then your sol so what does hacked me these are the when we think of hacked this is what we think of i actually did have a client call me and say not the site that i had built them luckily but a site that they were their brand was associated with was hacked by isis and sure enough there is this big you know we are Isis and this is what we believe in all sorts of highly inflammatory things on their branded website so this is kind of what we think of but it’s also denial of service it’s a data breach it can it doesn’t necessarily have to be somebody getting in and changing your front page or changing data on your page it can be as simple as just pulling small bits of information over time that will that will slowly leak in I was a bigger issue yes so defacement is actually in some ways that happens you go there and you know and from a sample size of three if your homepage is black and it’s not supposed to be if there’s a really creepy joker and then like a general like Guy Fawkes on your homepage is like a general sign you chances are that’s not your site so the next is defense in depth security as you guys may well know is not just a single level approach it is a multi-level approach and you have to go at it from

many many different angles so this de spans the gamut from from kind of that you know everything from the your hosting solution and the network there to the physical machines are using the OS a lot of the tools you’re running on the OS your web server your database server up through the JavaScript even to the team you work with to make sure they’re aware of security maybe you’re using a CDN so there’s a lot there to cover and i think one quote i love or one way to think about this i love is like the belt and suspenders approach you’re like you know this belt works pretty good these offenders vendors are pretty fly but I think with both of them I’ll be like very sure they kinda pants are gonna fall so you know you don’t just want to get one control you want to have as many controls over the whole stack and as we were talking about earlier it doesn’t matter how strong everything is if you have a weak link more audience participation how many of you have had somebody email you and say hey here’s the root password to my server yeah a client that owns their own owns their own server says hey I just spun it up on godaddy here’s the route to it you can be as as hardened as you want and that one email can just break everything apart so make sure that you you know don’t allow root access via password don’t even allow root access don’t even use root use of your own user harden it with with keys and we’ll talk about this later and and just go through some basic steps of and and part of this like that example is you have to teach the end users who aren’t going to the security conscious ones all of us know okay we don’t use root we don’t pass it around but you really need to tell your client when they send you that email look you probably shouldn’t do this you’ve just cause me a lot more issues than you’ve solved so so are you vulnerable are you vulnerable right now that’s a good question and it’s actually kind of hard to answer there are so part of the you know if you don’t know if you’re vulnerable it’s going to be hard to really you know get an actionable plan for for mitigating those risks so there are a couple different ways you can that you can kind of keep up to date on security updates so great one is us-cert which is a federal entity which kind of helps on aggregate and ice emanate information about this but you’re also going to want to keep up to date on kind of like any mailing list for any software using maybe that’s the patchy web server engine X maybe that’s my sequel Maria DB maybe that’s varnish or squid or anything like that also depending on your OS right you’re going to want to follow your OS security mailing list and maybe that’s what are a bunch of you and whatever that is but there’s like a you know so there’s like a couple great resources but really there’s no one great place where you can hear if there is a if there’s an exploit that might affect you and I think what one of the best tools I found for this is Twitter and it involved you know just like following you know the people i follow but that’s often i found out find out about exploits on twitter before you know us-cert kind of gets to it there may be a couple days days behind maybe before you know it’s kind of gone to the mailing list or something like that so and what other good thing is follow your own twitter handle from a different twitter handle so that you can see if somebody’s actually using yours so and if you’re a drupal developer the security team is awesome they have a full list of rundown of every security update decor what was affected how it’s affected the severity of it you’ll see all the information you need there to make the decision on how does this affect you and what do you need to do so we’re just going to switch gears a little bit and just talk briefly about compliance what what’s interesting about compliance is oftentimes organizations fall under multiple regulations I’ve had a lot of conversations here at drupalcon with universities and if you think about universities actually let’s see how many people are here from her for my college or a university wow that’s awesome you guys are like just many villages you guys you take in student data you have faculty data you have often have donor information you know you also probably have health centers for the students so you got HIPAA you have your state privacy laws your federal privacy laws are really lucky or maybe unlucky as a developer you could kind of work on a site that might hit olive oil yeah if you work at a state schools you probably do so um you know additionally i mentioned this earlier if you’re drooping developers you should also be asking your clients if they fall on your compliance regulations because sometimes they just don’t volunteer that and then you know towards the end of the project it’s like oh by the way oh yeah we have to do this we get that done in a week so

a couple chuckles people have heard everything so what what’s interesting about compliance requirements is it’s up to you or well it’s up to you and ultimately up to your clients but PCI pretty explicitly says that no hosting provider can actually make you compliant I think that that’s often a misunderstanding within developers a hosting provider can provide and it’s called an au pair aoc and attestation of compliance for their for their platform but that doesn’t extend to you and I’ll want to say that again because it’s really important that does not extend to you you need to put the proper controls in place you need to be doing the encryption you need to be doing the key management you need to be doing all the security things your hosting provider they can set you up to succeed but ultimately it’s up to you and I like it it to you spend all this money on building a armored vehicle but you don’t want to roll around with the windows down right just because you have this platform that is the most secure thing around it’s it’s not inherited that you’re going to be secure it’s up to you to make sure that you’re using it properly to be secured yeah I like the idea of offense phrase does shared responsibility and you know I think that comes back to that security is a responsibility it isn’t something you can you can you know do at the end of the project or isn’t something you can buy or isn’t something you can you know like a module you can install single module and just this further regulations like PSS make it very very clear that in the event of the breach it is the enterprise not the hosting provider or development agency that’s responsible so it’s also I mean you have reputational risk because you’ve developed that site but ultimately it is your client so I always find this slide particularly interesting because PII or personally identifiable information is not just a credit card or social security number I mean those are the obvious ones right but but like I mentioned earlier things like your login name your email address I mean gosh even if you’re collecting IP addresses as websites become a lot more personal they know who you are they’re collecting this information that stuff needs to be protected and yeah well we can just keep going here cool so um a little another little exercise so everybody raised their hand up and then we’re gonna lower it even you and then you as well everybody one of you okay cool so now lower your hand if you live east of the Mississippi River okay I ran the numbers and this is gonna work out perfectly lower you your hand if you do not own a dog yeah lower your hand if you are not born in February yeah that guy those two in the back in the back stand up so we went from like I don’t know how many people are in this room maybe 200 to like two people with like three totally like random random pieces of data that you wouldn’t really worry about but in this group if we if we had that we could identify those guys kind of it so that’s even stuff like you know your zip code date month of birth the state you know whether you like Coke or Pepsi more can kind of help like like target you and so that’s that’s kind of what you know good ad agencies are doing I guess but it’s also what good um you know good hackers are able to do with seemingly inconsequential pieces of information there was a there’s a high-profile instance of this where even data that you think is secure or that you think you’re you know the services that you’re using using our keeping secure there was a guy who I believe he was an editor or engadget or wired one of the major tech blogs had a very short one word twitter handle worth a lot of money and so somebody called up amazon and amazon said well I’m sorry can you answer your security question he kind of be asked around with him they said okay well can you give me the last four digits or the the security code on the card that ends in one two three four and he goes uh and then hung up call to apple and said oh and apple said well can you just verify the last four digits of your credit card which he just got from amazon so he’s able to verify it hacked into the iCloud account deleted everything on the guys and computers all of his family photos everything he had so this is something that is actively happening and it’s something that you have to be aware of even in from from as you’re developing standards and practices within your company’s what data are we going to give out and how are we going to give that out and could that data be then used to build a social hack on somebody sweet so we’re going to go into a couple things that you should absolutely be doing and

and if you’re not doing them it’s totally fine just to get up from the room you go or tisk our estate on it but but but these are the things if you do nothing else you have to be doing these the first one is back up your data I can’t I can’t stress this enough backups are going to save you in the long run no matter how you do it the I mean an example that I give is I have a newborn and i was up at you know five thirty in the morning feeding him and I’m sitting there and he’s sleeping in my arms and all of a sudden I get a ping from pingdom that your site is down I go and I look at it and something catastrophic that the content admins had added in cause all these issues and whatnot and rather than trying to deal with it with a sleeping child in my hand i was using pantheon at the time but i logged in and within two or three clicks I had a restore to back up the site was back up and running and my child was still sleeping so in that instance of backup saved me not only in the issue with the client but it also saved me some personal time trying to get my son back to sleep so just back up your data store it somewhere store it somewhere safe don’t store it on the same place where all your other data is because if all your data is breached or you lose it then your backup is worthless so keep it off-site keep it somewhere store it you know wherever you can and there’s tons of services available that will allow you to do that in conjunction with that you should be using some sort of source source code management and everyone here hopefully is using git if you’re not you know get with the program pun intended but get allows you to in the event of a breach one of the first things you can do and we’ll talk about this later is as you react by loading a backup and reverting your git repo that’s going to clean up a lot of your issues it’s not going to clean up any everything but it’s the first step that you can do and if you have get you know what files have been changed where they’ve been changed and how they’ve been changed if you can’t if you can’t identify that then you’re going to have to go line by line by line through the entire Drupal core and figure out where that little exploit is whereas if you can hit one button and get revert you’re going to be in a much better position yeah so another one is just use secure passwords right there are tools that can help with this weather it’s one password or LastPass or keep sx if you’re on linux and this is kind of XKCD joke about this which I don’t even well it’s kind of funny but really I think just use you know one login LastPass let you just create random passwords right this is just something you should be doing and something you should be doing for yourself it’s something you should be doing for your family and it’s something you should be doing for the team you work with right there are I’ll go into some slides about like the ssh attacks we see on Pantheon and like we are not like we use passwords we know your certificates but like there are just thousands of attempts like continuously and that you know that goes across the web you also want you know when gizmodo gets hacked you don’t want your password they are to be the same as your password for like Amazon or something like that so just take you know educate yourself and your team on basic the best best practices one strategy i use personally for that is I have a ultra unsecured password that I give to every site that I don’t even care about and that way you know if that thing gets hacked great you’ve just accessed some site that I’ll never access again and it has no information about me to you know layer your passwords use different passwords and it’s all about entropy don’t use you know your wife’s name or your dog’s name or your birthday or password 123 use something that has enough entropy in it that’s going to protect you and your your your environment entropy so another one is two factor auth so not every service supports this but every one that does that you use you should be using this you might think it’s a little bit of a pain and it is a little bit of a pain but it’s much less of a pain then you know like having your stuff hacked or your you know identity taken or something like that and once you get in the swing of it kind of like everything else it’s really easy to use at Pantheon we use yuba keys which are the ones on top left I’ll you know Google Authenticator RSA auth II there’s like a bunch of great options I can help introduce someone to any of those yeah and just to kind of elaborate on that a little bit two-factor authentication is something you know and something you have if you ask let’s say your mother’s maiden name and what was the name of your first pet that’s not two factor authentication that’s one factor twice so just I want to make sure everyone’s clear on that and and just to show the importance of two factor authentication everyone probably knows about the target breach that was due to stolen vendor credentials right you think at least I

do it’s very plausible that that break breach could have been avoided had they been using two-factor authentication because those stolen credentials would have just been like okay now where’s the pen and this all goes to speak to your not alone and from a Drupal site your Drupal site is no longer alone you know back five ten years ago your site was just your site it was very self-contained now your site is just a cog in the machine that that you are building weather for your client or within your your own business you’re you’re accessing marketing data you’re accessing internal data you have your internets your your marketing sites you’re going out you’re authenticating to external services authorize.net fedex you know paypal notes squirrel YouTube MailChimp you relic all of these are being accessed and they’re all trying to do that securely and you need to make sure that not only is Drupal being secure but you’re securely walking around and talking to these other services and if you think of Drupal’s role in the future of the web it will even be more so of that you know the hub integrating with all these different services so this will be an important security aspect to run forward and you don’t want and I’m sure a lot of you guys have had this we’re just because one service if you look at the COG if one service is attacked or one service is breached you don’t want to expose everything else in your system and it goes back into your daily life if you have one website login that’s breached then all of a sudden you know you have to go change your gizmodo here and got you have to go through all of your different sites and change them it’s because we no longer live in an isolated web we live in a very connected web cool so we’re gonna we’re about halfway through we’re gonna go and just some tips of securing your stack these are going to be really high level and not super super kind of action oriented but just kind of want to give you idea of the you know what you should be thinking about maybe a couple little tips and you know this is both for you and your colleagues how you can think about security so I’m happy to go into more depth and kind of references and stuff with anybody afterwards but I think we’ll kind of just haul through these so to me it starts like I’m biased I work at a hosting provider website management platform but I think that’s like really one of the one of the earliest points in the in the life cycle where you start thinking about security so there’s a lot of the stack kind of broken out into a couple parts hosting operating system down on the line but to me when you’re evaluating hosted hosting hosting should have security should absolutely be a criterion for evaluating hosting you also want to know what the strengths of your team is you know if you have a lot of like strong DevOps you build people who understand security of that security mindset already you know maybe you can roll your own a little bit more that’s not a strength of your team you know you want might want to look at a host can kind of compliment you there you don’t want to waste it as a dev shop you don’t want to waste time doing something that isn’t in your wheelhouse I’m not a host I don’t do that so why should I take on that role and waste my time when I can go be doing better things that’s can make me happier my client happier in the product better another another thing to know kind of picture of a typical corporate data center and so a lot of people who work in kind of bigger orgs and like there is a security team and they might be tempted to be like oh well the security team deals with that well that’s not the case same thing only you can educate yourself if you’re in this environment you might have a you know some new product something little tiny fluffy marketing brochure where site but that’s in the corporate firewall and this has been on an exploit that people have utilized before to get into the corporate firewall and get into your entire business through the stupidest thing that nobody cared about but was running in your in your corporate environment so that’s another thing to think about what else have a server what else is running in that Network securing your OS right so fedora bun to whatever it is the best thing you can do similar to drupal install the security updates right there are people like this is how you leverage the smart people that are working on security all the time you just apt-get install like yum upgrade whatever it is really you want to go for a sensible configuration this is kind of about the integration of risk to you don’t need to lock everything down but a lot of the defaults on your server really not be helped might not be helping you out that much a couple things I’d really look into our IP tables SSH please please don’t use passwords use certificates it’s just going to be way better and and lock down your soo doors so you know if you like even you know we all maybe get a little little buzz when we login and we get food but if you’re logging in regularly as root or logging in and regularly using like sudo su or something you are doing this wrong and you are like

leaving a vector open similarly engine X and apache this these like your web server you know you should love your web server and so we’re web server is going to be one of the quickest places to add or remove headers lockdown stuff like X frame options I think I’ll show some slides from some of pantheons logging and doing another session in tomorrow around elasticsearch which we use for logging but logs are super important right if you get hacked or you sense that there’s attack going on you want to understand how they were able to get in or where they were coming from so also take a keen eye on your web server to your access logs error logs that kind of thing that can go a long way to kind of understanding what happened and help make you feel a lot less kind of vulnerable in the event of a breach yeah a couple other a couple other little tips my sequel is awesome Randy V is awesome we all love them Drupal you know Drupal loves my sequel Maria DB as well the defaults also don’t help you out right change that root password lock down so it’s not accessible from other hosts this is a case my sequel is great because it’s really easy to get started playing with right but you know like that kind of comes at the cost of security a little bit so question the defaults educate yourself there’s you know handful five things you can do that will help really lock it down user root password root is not a good idea so I’ll just talk a little bit about data encryption currently within Drupal there is no native way to encrypt however there are a handful of modules and you can see them up there on the screen the thing you need to be really aware we touched on it earlier is you need to make sure that that key is taken care of but but luckily there are a lot of modules depending whether you need to encrypt your username your password form encrypt you know you can come talk to us after after the talk here sounds of security sponsors a lot of these so but leading into encryption key management that is the hard part of security it’s often said that encryption is the hard part of security and encryption key management is the hard part of encryption although it doesn’t really have to be because hackers don’t break encryption they find your keys so it’s just really a bad idea to have the keys to your kingdom tape essentially to your front door and that’s what you’re doing if you’re leaving your keys within the Google database settings file and fairness maybe that’s like under the front door mat and it’s like not page but it’s like you know if you did get in if you don’t wear it right weirdo yeah but and there was a lot of people here talking about compliance compliance regulations require key management and there are standards and best practices for key management defined by NIST which is the national institute of standards and technology and from our perspective key management is fundamental to a defensive depth approach to data security so talking about protecting API keys we talked about this earlier don’t share your API keys with your developers one of the things that that I regularly say is if you have a disgruntled employee do you really want them having access to your MailChimp account that they can send out email as you these are steps that you can take in all of these you can create you know fake accounts developer accounts and give those API keys out but keep the secret keys to yourself keep them secret don’t just pass them around to every developer that you have and don’t let your developer cherkis either make sure you have enough keys that they can all use it’s just like sharing a password so Drupal itself obviously keep it updated with with Drupal get and all that that came up last year it those security updates in the security team they know what they’re doing and when they publish a security update take it as as a necessity don’t just you know look at it saying I don’t think I’m going to do that greg did a great madison did a great talk yesterday on drupal security more in how to secure your site in your code definitely go look that up if you weren’t there and go check it out the other thing is you’ve got Drupal or a little feedback and Wolfers was about to get bad yeah cool just like low building tom is coming up here so Drupal core is great you can follow all the security updates that they have but then also know drupal site is built just around Drupal core you have all these contributed modules and you’re basically grabbing somebody else’s code and saying I trust you to

come and put code onto my server and into my site so you need to look at the look at the module is it active are the people working on it when was the last update who is the maintainer do you trust their code have you seen them work out of other places if you see a random module from a random developer that does something that you don’t really know don’t install it on your site and if you do see something like that you can raise an issue to you about security issues and the security team does look through those yeah so I think the last part of like the technical stack isn’t very technical it’s actually your team and so thinking about like strong passwords like it doesn’t matter if you have the strongest password but like in the world and it’s this long and you feel you know you have to type it out every time and then like your marketing in turn has you know like no password and that’s where they come in right so it’s about building that security consciousness not just for you but for your team sometimes you can enforce strong passwords through password complexity or enforce you know two factor auth or kind of enforce it like kind of you know enforce security but you also want to couple that with kind of just building awareness and building that consciousness and talking about security with kind of ever so real quick and we would do want to leave some time for questions if there are some hopefully you guys are thinking about this and have some questions but we want to tell you some real world examples of where all of this has played out in in our example or in our you know professional lives yeah so I think like we all kind of have some war stories and like we have a bunch and I’m sure ever like people in the room have some like you know interesting ways they were hacked or thought they might be getting hacked or trapped like you know attacks they were seeing whether it was you know adidas or kind of you know stuff like a base64 decode eval that they found in their code base or something so love to talk about that that stuff a little bit pre-fund just going to dig into a couple things so drupal get in and also pointing out that Matt’s giving a talk in this room at 5pm and it will be an awesome security talk about Drupal gannon specifically but this is like some of our pin from our pantheon blogging system when we we were able to kind of isolate you know the signature of the dribble get in attacks and and logged them and prevent them from having an impact so we saw that within you know several hours of the exploit being published we saw just going from A to Z across the platform you know alphabetically down the domains the attacks coming in and the attacks that weren’t on our site that were in somebody’s alphabetical list were like hitting you know Go Daddy Iraq we are anywhere else right so pretty like you know that gave me like complete faith in that like you know this is like a targeted attack and you need to get control of it we talked about SSH attacks a little bit so like the Internet is a great space and people are kinda like jiggling doorknobs and a tool i like is called fail2ban that locks out people who are tempting but you know if you have servers that are accessible to the internet they are just getting just continuously attacked and you could look in your SSH logs for that and it’s generally like this is why you don’t want to use a password you know it’s just that you’re just leaving leaving open a vector where somebody doing something really DOM is able to get in we’ve seen a couple targeted kind of H GDP denial of service attacks so these are kind of generally for kind of activist sites sites with kind of a political or social message one of them was around a this band called riot that got in some trouble after making fun of Putin and so we were hosting an activism site for that that was getting slammed and these aren’t super sophisticated tax kind of script kiddie but all the more so you don’t want that kind of script kiddie attack to take down your website and a good way to prevent those is put a CDN in front that’s going to be taking the brunt of that attack and so your your server your database and your lamp stack isn’t the one that’s taking on the full force of the deed Oz yeah there’s a couple CDNs in expo hall i think and stuff so definitely if if you do find yourself on the more might be a target side of the spectrum yeah i probably want to go here and they can fill you in on how they can help for myself I’ve actually been handed a database not once but multiple times that had thousands of credit card CTV’s expiration dates I had a client call me up and say hey there’s a site that you didn’t build but it’s in our system and we totally forgot to tell you about it but it’s storing all of these credit card and we just had a donor call us and say you just emailed my gmail account with my full credit card number thank you very much and so you know I’ve had to deal with this multiple times another thing that I really recommend you doing as a developer is stay in

touch with your with your clients I gave the clients the ability to create web forms and then I came back when they were having some issues and they called me up and they said hey we’re having some issues with a homecoming web form we have I go to it and they’re taking in credit card number ccv expiration date all that because they wanted to process tickets for their homecoming event and I had to kindly explain to them that you are using this completely wrong and you’ve just really caused a lot of issues so in what you’re doing especially there’s a lot of e-commerce everyone wants to do ecommerce everyone wants to take a payment somewhere online now do it safe to it securely do it right drupal commerce is a great great great platform to build off of but it itself is not just the key you have all these other pieces that build on top of it you can be running drupal commerce and you can be running it in securely so make sure you protect your credit card data and and part of this is just notifying your clients and educating them as well and saying look this isn’t a best practice you probably shouldn’t just store credit card numbers on a database somewhere and please don’t ever send those to me again like if we have time about this earlier in even up until walking away right if you don’t feel like you are empowered enough to you know to communicate that to your client like that might not be a place you want to be when you get that database of credit cards and I’ve actually done that I’ve walked away from a client because it’s just like I can’t touch this this is way too vulnerable way too much for me and I just want to expose myself and and in my brand I guess there’s my company to the level of insecurity that you’re willing to accept so just be be knowledgeable of it and try to educate your clients as much as you can so risk mitigation i mentioned that’s one of the other main reasons aside from compliance that i have a lot of conversations with members of the Drupal community and just the security community at large no one wants to see their names in the headlines for a data breach I think Sony had a hard time with that that was not good but there’s going to be brand damage there’s going to be loss of customers and loss of jobs it’s not just for a company it’s not just having to do credit monitoring for for all of their customers there’s just you see that they’re the stocks are going to tank and it’s just not not a good experience as a whole front for businesses so I mean you just got to do the right thing and i’ll use a case study here we recently partnered with somebody that was working with a international hotel chain at the hotel chain said we need someone to build us an Internet we’re going to have employee data in there and for anyone to even bid on this project there has to be encryption processes in place and so luckily that the partner came to us and was like is this something that you guys can help us with because we want to win this bid and we also need to make sure that we can you know not have our name in the headlines for when this site good win or if the psychic attack we need to protect the statement and so luckily we were able to protect them but I think it’s just a great example of someone being proactive with data security so last we want to leave you with okay you got hacked everyone this is their greatest fear they wake up and all of a sudden you realize this has happened to you it’s no longer something that you’ve learned about it’s no longer something that you’ve read about it is actually happening to you don’t panic react first thing roll back roll back your code roll back your database go back to a place where you know is secure and once you get the the site secure review the the best thing you can do because again we’re all going to get hacked at some point the best thing you can do for me to say why did I get hacked and how can i prevent that from happening again and and review that with the community and and reach out to other folks if you have no idea and you’re like I got hacked my website got two faced and I have no idea what happened go find somebody because it’s going to happen again unless you know what you did you know what you did wrong and that’s correct so that’s that’s a good point is that if there are processes in place that if you’re an academia that you have to you cannot immediately roll back just unplug it take it offline do whatever you can if you’re not in academia and you have a just a client facing side or a brochure site roll back as fast as possible and and basically the first step is protect yourself and protect the site do whatever that means if it means you

running down to the server room and just yanking the cord out of the wall then do that because you don’t want to allow it to continue to happen that’s a good point thank you so that’s kind of the end of what we’ve got here we do have some time for questions if anyone has any questions or comments or war stories or anything like that there’s a microphone in the middle if you feel comfortable using it because this is recorded so other people on the the webs can can hear about it hi hi I’m question so we have a website and there’s a lot of interactivity going on people signing in they put in their information as far as we know they’re as secure as we can make them everything you guys have mentioned we do but there’s one caveat when the development life cycle proceeds we need to take that database and bring it down to dev QA and stage in doing so we do a my sequel dump and that means it includes that PII direct what’s your strategy in allowing for that sdlc to continue but mitigating that risk where when you refresh dev QA and stage with a dev from production it precludes PII so from from a Drupal standpoint what I do is and and I do this on multiple reasons one is PII the other is I just don’t want to have a dev all of a sudden send an email out we’ve got a site that’s got a couple million emails on it I don’t want to all send blast them with a dev email so there are ways that you can script it so that on that dump or immediately after that dump happens before it goes back into the other environments that you go through and just scrub all of your data get it all out null it out do whatever you have to do and automate that process so that it’s not up to somebody to click a button and and do that right so really quick just to follow up on that so we we thought about that and in the my sequel dump we thought well we could write a script that completely or actually during the my sequel dump will take two dumps right first one is schema only the second one has ignore tables correct and but as soon as you have ignored tables and they’re emptied now those tables have relationships with other tables that PII once it’s empty those relationships are gone now your development environment dev QA and stage are useless yeah that’s tricky and I think one thing I’ll point out is right like dev in stage are two different things right so you might want to just have staging be kind of a more access with more more like live data you know you could instead of just ignore tables you could pipe that through a script which you know which kind of just jumbled up email addresses or you know something that you might consider sufficient but I think that it that is a tricky that is a tricky thing it’s kind it’s kind of a fun it goes right off and it’s a one-off that it depends on the data that you’re pushing and pulling if the data can be jumbled and it doesn’t break any relationships or if you then have to follow the train and make sure that you do the same changing to everything then that’s something you can do as well so one of the things we do really easy we just change every domain on the email list to an internal that just does nothing so then you just got a bunch of random you know strings of beginning of email but you don’t have the internals but again that’s going to be very dependent on each environment you’re in question building off what the other gentleman said about you know he’s got a Yank to court as soon as he’s attacked um instead of roll back I would argue redeploy on new clean hardware someplace else or the new host and everything correct isolation is key and so if you’re in an environment like a Pantheon or knock we’re that has a easy flow for that and you’re not going to redeploy right away you can roll back you can you can copy the database back to your dev place or your dev site and then roll back but again part of the review process is you have to be able to to know what happened if you immediately roll back you’re going to lose some of that information that’s again one that goes back to like like using version controls so important because if someone was able to get code in that you know yeah the version control should have that all that alright so thanks for the talk it was great earlier in the talk it sounded as though you may have alluded to the fact you may be using some form of honey pots by having other accounts that you just keep out there that you don’t use or keep any information and do you in fact use them if you do do you have any recommendations and how effective do you think they are all right say that again the kind of question was round like honey honey honey pot accounts for like drupal Google users or kind of ssh or something yeah yeah I haven’t had

specific experience with that I don’t think that’s like ultimately that effective I think that might be kind of fun but um but not um but not probably a strategy actually to mitigate your risk I wouldn’t in most cases i can think of um but um one kind of thing that’s interesting there and you know i think part of the devops track on vick on the cultural aspects of DevOps and like do what you need to do to convince people like that this stuff is real and they should be thinking about it and if that means like somehow like oh you know having a little honeypot or like looking at visualizing the logs for the ssh attack or that just the authorization failures from watchdog or whatever else like not quite an effective mitigation strategy but like a cool way to be like look guys this is actually happening like we need a you’re ready to talk about this and think about this and here’s you know here’s the visualization of that so something i might be more suitable for like a high-profile site so i use honey pots just as a general you know keep the crud out but one of the things you do have to realize is that that honey pot takes processing to run and there is a very easily targeted DDoS attack where if you know a forum has a heavy processing behind it you can just spam that form and even if the honey pots catching those and not doing anything to your data it can still take your site down so the honey pots a good good place to go and it gives you some information but I wouldn’t solely rely on that looks like a guy over there has Tomic dad yes track the IP addresses that they come from you can start doing bands as well yeah that’s kind of a cool ways like get people to attack and then be like okay I just know you’re a malicious and I’m not going to I’m going to just block you with using the keys over passwords is there any tools that use for managing that but for like SSH or SSH or any and in keys like that like management so within within Drupal we just are it’s in beta right now we’re hopefully going to be going to our see this this week we have a key module that were asking the community to start integrating into some of the more ecommerce and all those logos and stuff we’ve already got patches for a lot of those out there and basically it centralizes all the key management Drupal into one spots that you know where all those keys are part of the problem we’re right now is is in Drupal we treat those keys you know you’re using form API or something like that you can use the password field to collect that data and it gives you the nice pretty dots but what happens is that password just gets stored in the plane in the in the database and it’s getting stored either in your rules config or in your variable table so we’re trying that within a Drupal level to to centralize that key management and then from there there’s several options available to do that properly off-site and whatnot I’ll just add to that because that’s actually where my company lives is encryption key management so basically have with a lot of these modules that we’re sponsoring and that you can use to encrypt all the various data like we said it’s important to get it outside of the Drupal installation and not only just get it separated but you need to manage those keys so each time that you read and write it changes those keys so how we do that is just with these to be caller called hardware security modules now you can go to the AWS or azure wherever launch um and it’s just an encryption key management solution and we have a I leave the only integration with all of these Drupal modules so that’s that’s the way if you look at a lot of compliance requirements it’s going to say you need to be using missed validated encryption or fips 140-2 compliant key management based on industry standards which those are the industry standards and so you by doing that you’re able to meet compliance which goes back to the question we asked earlier about how many people needed to meet compliance this is a way to provably be meeting compliance and part of the updates we did to the encrypt module as well as allow for configurations before it was always one key 11 encryption method and when you change the key all of a sudden all the old data was just no encrypt now has the ability to support multiple keys and cycle the keys and do proper management there and ssh is easy and I can help you after if you want okay I mean like if you have a developer on your team and you have several different places where they use that key you have a central way to manage that key for a central way to manage the key in terms of like a mailchimp key or anything like that yes a your developer has access to 10 different 10 different things and they use the same key so yeah individually manages keys I use different keys so you know you can you can easily spin up a mailchimp we’ve done this a couple times

we just spin up a fake MailChimp account that just sends emails to us and in reality if you do all of your dead process around that and the last-minute switch the key you’re totally fine you know so that’s that’s one of the ways that you can manage that and we’re we’re also working on some solutions that will manage that as well thank you hey there sexy just more of a comment because I was thinking about the the backup talk about importance of that so just kind of experience that we ran into it’s worth sharing is that having backups from more than just like the last week and sometimes be important we with one of our clients who wanted to handle their hosting all in their own they weren’t making security updates to the Drupal platform and then they were hacked a month earlier yeah and then none of the backups on the platform we were there on had data that was further back than a week or two weeks and so having a roll back to be able to go back a month you know generally is something I would you know just put out there is yeah rolling back is not necessarily you’re not going to always know about it that day depending on who your client is so before it became very popular to do in the platform level and with options like note squirrel and stuff I actually had a bash script that would take daily backups and then every week it would take one of those and stored off as a weekly backup and they would keep you know five weeks of backups and do a monthly back up and then do a yearly back up so that you had a directory that’s like I could go all the way back one year and not you know you don’t you don’t store a large number of files you’re only storing like 10 at the most or 10 backups of the most but you can go back historically through and that’s a that’s a very good point cool I think that’s all the time we have real so real quick will answer your question that will run out of here so so I have some drupal accounts on shared hose grass and i was looking at a log yesterday the recent log entries and i’m starting to see a lot of page not found for things like WP admin which the Empress drag do you have any sense of what’s going on and so precautions the pake in that case so shared environments are very their Jeep and they’re easy but they’re also very insecure in that matter because a lot of the time somebody will just go to the server and then just start spamming everything on the server with logins and then it’s just them going around shaking door handles waiting for one to open and so I wouldn’t I would log it i would tell your service provider hey i’m seeing these come in and i’m seeing an increase of them but unless they’re actually getting at your data or if you have ssh that’s insecure anything like that then I think you’re going to be okay yeah we see that constantly not just people walk around the internet grabbing domain names jiggling the doorknob seeing and if it’s there looking for WordPress and your Drupal he’ll be all right yeah I see it happens all the time on on your on your grid service or anything like that I had I had a was at the there was a caching module and it would cash all the requests that came in and i looked at the cash log and it was just you know all these random sites and they were all just ones that were on a shared host with it so very common I don’t think get to worry about it thank you yeah thank you guys funny one in the first three rows for your clothes participation I have a quick present