Monday, August 29, 2016

How do we explain email to an "expert"?

This has been a pretty wild week, more wild than usual I think we can all agree. The topic I found the most interesting wasn't about one of the countless 0day flaws, it was a story from Slate titled: In Praise of the Private Email Server

The TL;DR says running your own email server is a great idea. Almost everyone came out proclaiming it a terrible idea. I agree it's a terrible idea, but this also got me thinking. How do you explain this to someone who doesn't really understand what's going on?

There are three primary groups of people.

1) People who know they know nothing
2) People who think they're experts
3) People who are actually experts

If I had to guess, most of #3 knows running your own email server is pretty dangerous. #1 probably is happy to let someone else do it. #2 is a dangerous group, probably the largest, and the group who most needs to understand what's going on.

These ideas apply to a lot of areas, feel free to substitute the term "security" "cloud" "doughnuts" or "farming" for email. You'll figure it out with a little work.

So anyway.

A long time ago, if you wanted email you basically had to belong to an organization that ran an email server. Something like a university or maybe a huge company. Getting a machine on the Internet was a pretty big deal. Hosting email was even bigger. I could say "by definition this meant if you were running a machine on the Internet you were an expert", but I suspect that wasn't true, we just like to remember the past as being more awesome than it was.

Today anyone can spin up a machine in a few seconds. It's pretty cool but it also means literally anyone can run an email server. If you run a server for you and a few other people, it's unlikely anything terrible will happen. You'll probably get pwnt someday, you might notice, but the world won't end. How do we convince this group that just because you can, doesn't mean you should? The short answer is you can't. I actually wrote about this a little bit last year.

So if we can't convince them what do we do? We get them to learn. If you've ever heard of the Dunning Kruger effect (I talk about it constantly), you understand the problem is generally a lack of knowledge.


You can't convince experts of anything, especially experts that aren't really experts. What we can do though is encourage them to learn. If we have someone we know is on the peak of that curve, if they learn just a little bit more, they're going to fall back to earth.

So I can say running your email server is a terrible idea. I can say it all day and most people don't care what I think. So here's my challenge. If you run your own email server, start reading email related RFCs, learn about things like spam, blacklisting, greylisting, SPF. Read about SMTPS, learn how certificates work. Learn how to mange keys, learn about securing your clients with multi factor auth. Read about how to keep the mail secure while on disk. There are literally more topics than one could read in a lifetime. If you're an expert, and you don't know what one of those things are, go learn it. Learn them all. Then you'll understand there are no experts.

Let me know how wrong I am: @joshbressers

Sunday, August 21, 2016

The cost of mentoring, or why we need heroes

Earlier this week I had a chat with David A. Wheeler about mentoring. The conversation was fascinating and covered many things, but the topic of mentoring really got me thinking. David pointed out that nobody will mentor if they're not getting paid. My first thought was that it can't be true! But upon reflection, I'm pretty sure it is.

I can't think of anyone I mentored where a paycheck wasn't involved. There are people in the community I've given advice to, sometimes for an extended period of time, but I would hesitate to claim I was a mentor. Now I think just equating this to a paycheck would be incorrect and inaccurate. There are plenty of mentors in other organizations that aren't necessarily getting a paycheck, but I would say they're getting paid in some sense of the word. If you're working with at risk youth for example, you may not get paid money, but you do have satisfaction in knowing you're making a difference in someone's life. If you mentor kids as part of a sports team, you're doing it because you're getting value out of the relationship. If you're not getting value, you're going to quit.

So this brings me to the idea of mentoring in the community.

The whole conversation started because of some talk of mentoring on Twitter, but now I suspect this isn't something that would work quite like we think. The basic idea would be you have new young people who are looking for someone to help them cut their teeth. Some of these relationships could work out, but probably only when you're talking about a really gifted new person and a very patient mentor. If you've ever helped the new person, you know how terribly annoying they become, especially when they start to peak on the Dunning-Kruger graph. If I don't have a great reason to stick around, I'm almost certainly going to bail out of that. So the question really is can a mentoring program like this work? Will it ever be possible to have a collection of community mentors helping a collection of new people?

Let's assume the answer is no. I think the current evidence somewhat backs this up. There aren't a lot of young people getting into things like security and open source in general. We all like to think we got where we are through brilliance and hard work, but we all probably had someone who helped us out. I can't speak for everyone, but I also had some security heroes back in the day. Groups like the l0pht, Cult of the Dead Cow, Legion of Doom, 2600, mitnick, as well as a handful of local people. Who are the new heroes?

Do it for the heroes!

We may never have security heroes like we did. It's become a proper industry. I don't think many mature industries have new and exciting heroes. We know who Chuck Yeager is, I bet nobody could name 5 test pilots anymore. That's OK though. You know what happens when there is a solid body of knowledge that needs to be moved from the old to the young? You go to a university. That's right, our future rests with the universities.

Of course it's really easy to say this is the future, making this happen will be a whole different story. I don't have any idea where we start, I imagine people like David Wheeler have ideas. All I do know is that if nothing changes, we're not going to like what happens.

Also, if you're part of an open source project, get your badge

If you have thoughts or ideas, let me know: @joshbressers

Monday, August 15, 2016

Can't Trust This!

Last week saw a really interesting bug in TCP come to light. CVE-2016-5696 describes an issue in the way Linux deals with challenge ACKs defined in RFC 5961. The issue itself is really clever and interesting. It's not exactly new but given the research was presented at USENIX, it suddenly got more attention from the press.

The researchers showed themselves injecting data into a standard http connection, which is easy to understand and terrifying to most people. Generally speaking we operate in a world where TCP connections are mostly trustworthy. It's not true if you have a "man in the middle", but with this bug you don't need a MiTM if you're using a public network, which is horrifying.

The real story isn't the flaw though, the flaw is great research and quite clever, but it just highlights something many of us have known for a very long time. You shouldn't trust the network.

Not so long ago the general thinking was that the public internet wasn't very trustworthy, but it all worked well enough that things worked. TLS (SSL back then) was created to ensure some level of trust between two endpoints and everything seemed well enough. Most traffic still passed over the network unencrypted though. There were always grumblings about coffee shop attack or nation state style man in the middle, but practically speaking nobody really took these attacks seriously.

The world is different now though. There is no more network perimeter. It's well accepted that you can't trust the things inside your network any more than you can trust the things outside your network. Attacks like this are going to keep happening. The network continues to get more complex, which means the number of security problems increases. IPv6 will solve the problem of running out of IP addresses while adding a ton of new security problems in the process. Just wait for the research to start taking a hard look at IPv6.

The joke is "there is no cloud, just someone else's computer", there's also no network, it's someone else's network. It's someone else's network you can't trust. You know you can't trust your own network because it's grown to a point it's probably self aware. Now you expect to trust the network of a cloud provider that is doing things a few thousand times more complex than you are? You know all the cloud infrastructures are held together with tape and string too, their networks aren't magic, they just have really really good paint.

So what's the point of all this rambling about how we can't trust any networks? The point is you can't trust the network. No matter what you're told, no matter what's going on. You need to worry about what's happening on the network. You also need to think about the machines, but that's a story for another day. The right way to deal with your data is to ask yourself the question "what happens if someone can see this data on the wire?" Not all data is super important, some you don't have to protect. There is some data you have that must be protected at all times. That's the stuff you need to figure out how to best do something like endpoint network encryption. If everyone asked this question at least once during development and deployment it would solve a lot of problems I suspect.

Monday, August 8, 2016

We're figuring out the security problem (finally)

If you attended Black Hat last week, the single biggest message I kept hearing over and over again is that what we do today in the security industry isn't working. They say the first step is admitting you have a problem (and we have a big one). Of course it's easy to proclaim this, if you just look at the numbers it's pretty clear. The numbers haven't really ever been in our favor though, we've mostly ignored them in the past, I think we're taking real looks at them now.

Of course we have no clue what to do. Virtually every talk that touched on this topic at Black Hat had no actionable advice. If you were lucky they had one slide with what I would call mediocre to bad advice on it. It's OK though, a big part of this process is just admitting there is something wrong.

So the real question is if what we do today doesn't work, what does?

First, let's talk about nothing working. If you go to any security conference anywhere, there are a lot of security vendors. I mean A LOT and it's mostly accepted now that whatever they're selling isn't really going to help. I do wonder what would happen if nobody was running any sort of defensive technology. Would your organization be better or worse off if you got rid of your SIEM? I'm not sure if we can answer that without getting in a lot of trouble. There is also a ton of talk about Artificial Intelligence, which is a way to pretend a few regular expressions make things better. I don't think that's fooling anyone today. Real AI might do something clever someday, but if it's truly intelligent, it'll run away once it gets a look at what's going on. I wonder if we'll have a place for all the old outdated AIs to retire someday.

Now, on to the exciting what now part of this all.

It's no secret what we do today isn't very good. This is everything from security vendors selling products of dubious quality, to software vendors selling products of dubious quality. In the past there has never been any real demand for high quality software. The selling point has been to get the job done, not get the job done well and securely. Quality isn't free you know.

I've said this before, I'll keep saying it. The only way to see real change happen in software if is the market forces demand it. Today the market is pushing everything to zero cost. Quality isn't isn't free, so you're not going to see quality as a feature in the mythic race to zero. There are no winners in a race to zero.

There are two forces we should be watching very closely right now. The first is the insurance industry. The second is regulation.

Insurance is easy enough to understand. The idea is you pay a company so when you get hacked (and the way things stand today this is an absolute certainty) they help you recover financially. You want to ensure you get more money back than you paid in, they want to ensure they take in more than they pay out. Nobody knows how this works today. Is some software better than others? What about how you train your staff or setup your network? In the real world when you get insurance they make you prove you're doing things correctly. You can't insure stupidity and recklessness. Eventually as companies want insurance to protect against losses, the insurance industry will demand certain behaviors. How this all plays will be interesting given anyone with a computer can write and run software.

Regulation is also an interesting place to watch. It's generally feared by many organizations as regulation by definition can only lag industry trends, and quite often regulation adds a lot of cost and complexity to any products. In the world of IoT though this could make sense. When you have devices can literally kill you, you don't want anyone building whatever they want using only the lowest quality parts available. In order for regulation to work though we need independent labs, which don't really exist today for software. There are some efforts underway (it's an exercise for the reader to research these). The thing to remember is it's going to be easy to proclaim today's efforts as useless or stupid. They might be, but you have to start somewhere, make mistakes, fix your mistakes, and improve your process. There were people who couldn't imagine a car replacing a horse. Don't be that person.

Where now?

The end game here is a safer better world. Someday I hope we will sip tea on a porch, watching our robot overlords rule us, and talk about how bad things used to be. Here's the single most important part of this post. You're either part of the solution or you're part of the problem. If you want to nay-say and talk about how stupid these efforts all are, stay out of the way. You're part of an old dying world that has no place in the future. Things will change because they must. There is no secret option C where everything stays the same. We've already lost, we got it wrong the first time around, it's time to get it right.

Monday, August 1, 2016

Everyone has been hacked

Unless you live in a cave (if you do, I'm pretty jealous) you've heard about all the political hacking going on. I don't like to take sides, so let's put aside who is right or wrong and use it as a lesson in thinking about how we have to operate in what is the new world.

In the past, there were ways to communicate that one could be relatively certain was secure and/or private. Long ago you didn't write everything down. There was a lot of verbal communication. When things were written down there was generally only one copy. Making copies of things was hard. Recording communications was hard. Even viewing or hearing many of these conversations if you weren't supposed to was hard. None of this is true anymore, it hasn't been true for a long time, yet we still act like what we do is just fine.

The old way
Long ago it was really difficult to make copies of documents and recording a conversation was almost impossible. There are only a few well funded organizations who could actually do these things. If they got what they wanted they probably weren't looking to share what they found in public.

There was also the huge advantage of most things being in locked building with locked rooms with locked filing cabinets. That meant that if someone did break it, it was probably pretty obvious something had happened. Even the best intruders will make mistakes.

The new way
Now let's think about today. Most of our communications are captured in a way that makes it nearly impossible to destroy them. Our emails are captured on servers, it's trivial to make an infinite number of copies. In most instances you will never know if someone made a copy of your data. Moving the data outside of an organization doesn't need any doors, locks, or passports. It's trivial to move data across the globe in seconds.

Keeping this in mind, if you're doing something that contains sensitive data, you can't reliably use an electronic medium to transport or store the conversations. emails can be stolen, phone calls can be recorded, text messages can be sniffed going through the air. There is almost no way to communicate that can't be used against you at some later date if it falls into the wrong hands. Even more terrifyingly is that an attacker doesn't have to come to you, thanks to the Internet, they can attack you from nearly any country on the planet.

What now?
Assuming we don't have a nice way to communicate securely or safely, what do we do? Everyone has to move information around, information is the new currency. Is it possible to do it in a way that's secure today? The short answer is no. There's nothing we can do about this today. If you send an email, it's quite possible it will leak someday. There are some ways to encrypt things, but it's impossible for most people to do correctly. There are even some apps that can help with secure communications but not everyone uses them or knows about them.

We need people to understand that information is a currency. We understand the concept of money. Your information is similarly valuable. We trade currency for goods and services, it can also be stolen if not protected. Nobody would use a bank without doors. We store our information in places that are unsecured and we often give out information for free. It will be up to the youth to solve this one, most of us old folks will never understand this concept any more than our grandparents could understand the Internet.

Once we understand the value of our information, we can more easily justify keeping it secure during transport and storage. Armored trucks transport money for a reason. Nobody is going to trust a bicycle courier to move large sums of cash, the same will be true of data. Moving things securely isn't easy nor is it free. There will have to be some sort of trade off that benefits both parties. Today it's pretty one sided with us giving out our information for free with minimal benefit.

Where do we go now? Probably nowhere. While I think things are starting to turn, we're not there yet. There will have to be a few more serious data leaks before the right questions start to get asked. But when they do, it will be imperative we understand that data is a currency. If we treat it as such it will become easier to understand what needs to be done.

Leave your comments on twitter: @joshbressers

Monday, July 18, 2016

Using a HooToo Nano as a magic VPN box

I've been getting myself ready for Blackhat. If you're going you know this conference isn't like most. You don't bring your normal gear with you. You turn the tinfoil hat knob up to an 11, then keep turning it until it breaks off. I did do one thing that's pretty clever this year though, I have no doubt it could be useful for someone else putting together an overengineered tin foil hat security rig.

When I travel I use a little travel router from HooToo. Specifically this one. The basic idea is I can use either ethernet or wifi to connect all my devices to the Internet. I get my own private network behind the device which lets the Chromecast work in a hotel and means I don't have to keep logging in 15 devices once a day. This got me thinking though, wouldn't it be cool if the HooToo router could VPN for me.

Enter the HooToo Nano.

Now I'm sure I could have found a travel router someone makes that does VPN, but that's not nearly as exciting as figuring this out myself, bricking it a few times, unbricking it, and eventually having a solution that works well enough I can live with it. You can install OpenWRT on it which makes it an insanely awesome device.

Here's the basics. I connect the router to a wireless network (which is a pain to with OpenWRT). Once I'm connected up, I flip the switch on the side of the Nano and it connects to the VPN, a green light turns on once the VPN is active. Everyone knows green means good, right? If I flip the switch back, it turns the VPN off (the green light turns off). The biggest problem was there is a bug in OpenWRT where if one of the wireless networks it's configured to connect to can't be found, none of the wireless will come up. My solution is I can hit the reset button to return the router to a known good state.

In the spirit of open source, I'll explain how to do all this. Your mileage may vary, it's not simple, but let's face it, it's awesome. I have a magic box that when the green light turns on, I no longer have to worry about the scary local wifi. Perfect for a conference where nobody and nothing can be trusted.

On with the show.

First, you need a HooToo Nano (this is easy). Then you install OpenWRT (this is less easy). I'm not going to explain this part. Apart from already being documented, I don't want to do it again to write it down, I have things working, I'm not touching anything.

Next you need to get openvpn working on it. I followed these instructions from the IPredator folks.

At this point you should have a functioning VPN if you run the init.d openvpn script. With the VPN up, I setup a firewall target called 'vpn'. That name will be important later.

First, we will need to create a nice default configuration. As I said before, OpenWRT has a bug where if one of your wireless networks can't be found, none will work. As I don't have time to figure that bug out right now, I put together some configuration files that only have one wireless network configured as an access point. This configuration exists so I can connect to the router and setup more networks. I then copied all the configuration files from /etc/config to /root/config/
Then I edit /etc/rc.button/reset to add the line
cp /root/config/* /etc/config/
Right before the sync and reboot commands. By doing this I can hit the reset button with a paperclip to return the router to my default settings. Also as a side note, if you hold the reset button down for more than 5 seconds it will do an OpenWRT factory reset, so don't do that.

Lastly, we setup the switch. The best way I could find to read it was by creating the directory /etc/hotplug.d/button, then adding an executable script called "buttons" to it.
root@OpenWrt:~# cat /etc/hotplug.d/button/buttons
#!/bin/sh
. /etc/profile
#logger the button was $BUTTON and the action was $ACTION
if test "$BUTTON" = 'BTN_0'; then
if test "$ACTION" = 'pressed'; then
uci set firewall.@forwarding[0].dest='vpn'
/etc/init.d/openvpn start
fi
if test "$ACTION" = 'released'; then
uci set firewall.@forwarding[0].dest='wan'
/etc/init.d/openvpn stop
/sbin/fw3 reload
fi
fi
As you can see in the script, I set the vpn firewall to my forwarding target. If you name your vpn firewall something else, be sure to change it.

Without  a doubt these instructions aren't as clear as they should be. I don't have time right now to write this up properly, someday I would love to put together an OpenWRT image with all this baked in, but for the moment I hope it's useful for someone.

If you try this and have questions, feel free to find me on Twitter: @joshbressers

Monday, July 11, 2016

Entry level AI

I was listening to the podcast Security Weekly and the topic of using AI For security work came up. This got me thinking about how most people make their way into security and what something like AI might mean for the industry.

In virtually every industry you start out doing some sort of horrible job nobody else wants to do, but you have to start there because it's the place you start to learn the skills you need for more exciting and interesting work. Nobody wants to go over yesterday's security event log, but somebody does it.

Now consider this in the context of AI. AI can and will parse the event logs faster and better than a human ever could. We're terrible at repetitive boring tasks. Computers are awesome at repetitive boring tasks. It might take the intern two hours to parse the log files, it will take the log parser two seconds. And the computer won't start thinking about donuts halfway through. Of course there are plenty of arguments how today's AI have problems which is true. They're still probably better than humans though.

But here is what really got me thinking. As more and more of this work moves to the domain of AI and machines, what happens to the entry level work? I'm all for replacing humans with robots, without getting into the conversation about what will all the humans do when the robots take over, I'm more interested in entry level work and where the new talent comes from.

For the foreseeable future, we will need people to do the high skilled security work. By definition most of the high skilled people are a bit on the aged side. Most of us worked our way up from doing something that can be automated away (thank goodness). But where will get our new batch of geezers from? If there are no entry level offering, how can security people make the jump to the next level? I'm sure right now there are a bunch of people standing up screaming "TRAINING", but let's face it, that only gets you a little way there, you still need to get your hands dirty before you're actually useful. You're not going to trust a brain surgeon who has never been in an operating room but has all the best training.

I don't have any answers or even any suggestions here. It just happened to get me thinking. It's possible automation will follow behind the geezers which would be a suitable solution. It's possible we'll need to make some token entry level positions just to raise the skill levels.

What do you think? @joshbressers