It’s hard to understate how amazing DEFCON is. I met – or got to see speak – so many of my heroes. So many talks were mind-blowing and/or eye-opening. These people really are the best of the best.

Nearly every talk I saw had merit and value, and most of them were incredibly engaging and awesome!

By the end of Thursday they had over 12k people so I would imagine that this year’s attendance was in the neighborhood of 15k – it’s huge. Really huge. But awesome.

I’ve tried to summarize the talks and events I went to… hopefully it’s useful.

Thursday

First Day Bits

Spent 2 hours in line for my badge. Fortunately I ran into my friends Kevin and Vivienne as I got in line, so I had company!

Because of this I missed the talks I wanted to see – one I was in line for and the other had a line out the door to get into. I did some social things instead.

Spent some time with Kevin working on the Forensics challenge… made headway, but not a ton.

Facebook Student Recruiting Night

FB rented out a small club, invited 40 students, and paid for alcohol and food. In typical Vegas style, the cocktail waitresses were eyecandy.

Neil Poole gave a speech about be being an intern-turned-FTE and how amazing FB is, and then we just chatted with the interns all night. It wasn’t a pressure-to-come-work atmosphere, more of a, get-to-know-us-and-ask-us-questions atmosphere. It was fun, but the music got progressively louder throughout the night, so the last hour was a bit useless.

Friday

Domain Name Problems and Solutions – Paul Vixie!

I was very excited to see Paul Vixie’s talk on “Domain Name Problems and Solutions.” If you’re wondering where you recognize that name from, it’s because the name of cron is “vixie-cron” (which he wrote). He also founded the original ISC, wrote large swaths of BIND, invented RBLs, and is undoubtedly one of the fathers of the internet.

When I first left his talk I was a bit disappointed, but in hindsight this wasn’t because his content or his talk were bad (they weren’t), but instead that his initial premise didn’t sit well with me. However, that doesn’t make the rest of his talk any less valuable.

His initial arguement was that the cost of getting a domain name being “too cheap to meter” – i.e. you can buy one for very little with no validation of who you are and have globally live within 30 seconds – is broken. It is needless and attracts spammers and phishers. Because we don’t make it difficult to get a domain, the next best thing to fight bad actors – and where most of our effort has been focused – is shutting them down after they get the domain names.

However, registrars are getting rich because as soon as they delete a spammy domain name the bad guys buy another. Vixie doesn’t want to feed that cycle. So we’ve moved onto the third choice, solving this at endpoints – homes/businesses/etc.

Let me note here that I agree there is a grand amount of badness floating around in registered domains; I simply disagree that making them hard-to-get or expensive is a good solution. Other filtering could be done. Anyway…

He built a system that passively scans the world of DNS (DNSDB) and puts it in a database so you can now query the history of DNS in realtime. It allows wildcard searches and other things as well. So if you’re trying to figure out where an attacker sent you, but the domain name is long-gone, now you still can. This is free for non-profit use. Through this he did a lot of research and found things like… most domains don’t last more than 10 minutes, so if we can ignore domains for the first 10 minutes of their life, we’d be in a better place.

So he created DNS Response Policy Zones (RPZs) – a kind of DNS firewall. It’s similar to the RBLs, but for DNS instead of mail. So he has one, for example, that doesn’t resolve addresses from domains less than 10 minutes old.

All in all, the content of his talk was very good, and I’m excited to play with DNSDB. Also, I had the pleasure of getting to say hi and shake his hand after his talk which made my day.

USB for all!

This talk was the talk of the internet for weeks before defcon. These guys apparently found a generic way to reporogram USB firmware chips in nearly any USB device (keyboards, mice, memory sticks, etc.), to pwn any computer they are plugged into. However, this was in a tiny room and I was unable to get in.

How To Get Phone Companies to Just Say No To Wiretapping – Zimmermann!

This was done by Phil Zimmermann – the father of PGP. Talk about a day of meeting heroes.

I missed the first 10 minutes of this talk, which was apparently the whole actual “talk” part – the other 40 minutes was Q&A. I gleaned from this what he was talking about – he recently announced Blackphone. It’s a privacy-centric phone. It’s a very secure and stripped down version of Andriod, where the phone app has been replaced by an Silent Phone, an app that does a Diffie-Hellman key-exchange to get ephemeral keys, encrypts the entire conversation, and then throws away keys. But wait, there’s more! You can download Silent Phone for stock android too.

But it gets better. It supports making calls to POTS lines too. I didn’t catch all the details here, but it seems a central server at his company that will terminate the crypto in the event you are calling a POTS line, as close to the recipient as possible and then send it on. This may seem “useless”, but imagine you’re a dissident in Iran and calling a reporter in the US. There is no unencrypted part the Iranian government could get its hands on.

This is very cool.

I also didn’t catch the details on this part, but like a normal phone service and you need to buy a calling plan… but your call data is never stored on their servers (because it can’t be). However, it’s all standard protocols (ZRTP and SCIMP) which they are IETF proposals now. It’s unclear if a plan is necessary if always calling between Silent Phones, or only of wanting to call POTS lines. He said that while he believes ZRTP and SCIMP are high-quality, secure, and robuts protocols, if the world adopts a different one he’ll move Silent Phone and Blackphone to it.

My favorite quote, “I think public key infrastructures are generally a bad idea. I think they should be avoided whenever possible.”

Am I Being Spied On? Low-tech Ways Of Detecting High-tech Surveillance

This was a bunch of anti-surveillance techniques. Maybe I’m not paranoid enough, but a lot of these seemed obvious or silly. However, there were a few cool ones. The one that stood out the most was detecting hidden night-vision cameras. Night-vision cameras use infrared to “see” (light) which is picked up brightly by ordinary cell phone cameras. So if you turn off all the lights in your house and walk around filming with your cell phone you’ll see a big bright light on the phone if you pass a hidden night-vision camera. There were other good ones as well, but on balance, this wasn’t my favorite talk.

The Open Crypto Audit Project

This was really interesting.

You remember a few months ago when TrueCrypt up and vanished? They made the code only read TC volumes, said it was insecure, and then disappeared?

So these Open Crypto guys noticed that despite the very prevalent use of TrueCrypt across business, personal, and legal realms, it had never had an audit, so they started a kickstarter to get the code audited and they got TONS of money. So much more than they expected they realized they had to stop and figure out a more detailed plan, file for 501(c)3 status, etc.

They decided to proceed with two phases: Phase 1 would be a general secure-coding practices talk, no cryptanalysis at all. Just… do they do bounds checking, do they clear memory properly, etc. They hired iSec, probably the best company in the world for code auditing, to do this and while the code was far from perfect, it did better than most other software, despite having a few dumb mistakes.

The day after the published the very positive results, TC, without any warning, vanished. Emails bounced, domain names were unregistered. The code had been hosted on their own git repo for a long time, it was moved back to SF but in a form that would only read, not write – and since the previous SF checkin was from years prior before they’d gotten their own site up, there’s no code history.

OCA tried hard to track down those guys and find out what happened, but the developers they had built a relationship had no idea what happened either.

They’re trying to reconstruct the 7.1 source so they can proceed with the audit anyway, since it’s so widely used.

John McAfee “secret” talk

McAfee did a talk at B-sides the week before and there was a rumor he was one of the so-called “Sky Talks” (Sky Talks are often “TBD”, are never recorded, and are usually legally questionable in nature). I got into this one by happy accident – a co-worker had told me about the TCP/IP trivia drinking game, and I was in line to get into it. Turns out they had switched rooms and I ended up in the McAfee talk.

He discussed (all of this, of course, is his version of the story) what went down in Beliz, having men with guns show up at his apartment in Portland, life on the run, crazy cell-phone bugging that happens to him. He also discussed some of the implications of our always-on always-sharing society and that we need to raise awareness – while it’s good and it’s not going away, we need to help people make better decisions. He has a free app, DCentral 1, that will tell you what permissions every app on your phone has.

He’s also working on some other privacy and awareness based things.

His Q&A was very impressive. People had suggestions for him (like providing his app for direct download for side-loading on Android so you don’t have to trust Google Play) which he was always up for and made his PA take notes of.

I left his talk being a lot less sure he was crazy. As I said to everyone else, he could be nuts, or this could all in fact be really happening. Unlike most dillusional people I’ve known he doesn’t speak in a disjointed manner. He speaks cogently, with clarity, purpose, and clear timelines. Everyone I talked to was equally unsure.

Facebook Recruiting Party

Facebook threw a 400-person party at the Surrender Club at the Wynn. It’s a club with pools. And a strict dress code. Which is an odd juxtaposition.

Party was interesting. There were a lot of EFF folks there, which was cool, plus a ton of others. People seemed to have fun and enjoy themselves.

They wouldn’t let us in the main pool, and there was a tiny little pool not much larger than a hot-tub we were allowed to use. As you might imagine, this didn’t inspire people to want to swim. Only 5 people – including yours truly – swam at all.

I’d say the biggest thing I got out of this was some really good conversations with our head of security, Joe Sullivan. He’s a very interesting dude. He also gave me a lot of his private stock of tequila. 🙂

Saturday

The Internet of Fails: Where IoT Has Gone Wrong and How We’re Making It Right

This. Was. Eye-opening.

OMG. So … as you know we are entering the era of the “internet of things” – IPv6 lightbulbs and internet-enabled cameras and watches and fitbits and whatever.

Yeah, so it turns out these are all just… horrible. They looked at about 40 devices and most of them had easy ways to get remote shells. One of the most common ones was a root password baked into the mobile app used to configure the device.

The scariest ones were all the nanny-cams or other home-security cams. Almost all of them were trivially pwned. The worst of them was uploading the video to an unauthenticated S3 bucket, so anyone could see, for example, pictures of you getting out of the shower or whatever.

It wasn’t rare… it was nearly every embedded device they looked at.

Most of these devices are made through kickstarters where the makers don’t have the resources to spend much time worrying about security, but even the major brands had similar problems.

In order to help the situation, they setup BuildItSecure.ly which is two things. First, it’s an organization with partners amongst some of the larger makers of these things who will provide pre-production versions to them so their security researchers (who are volunteers) can test it. In return they get a ‘stamp-of-approval’ if their stuff passes muster. The second thing is a collection of curated resources on how to build these sorts of things securely.

Secure Random by Default – Dan Kaminsky!

Back to the meeting-my-heroes theme. Dan Kaminsky is the guy who did most of the work on reverse engineering the Sony Rootkit. He discovered vulnerabilities in DNS that led to DNS cache poisoning in most DNS servers, and is probably one of the most well-respected security researches in the world.

This was a very rare 2-hour talk. Dan… rambles a lot. He covered lots of things. He talked about hard-drives and he talked about entropy and a bunch of other stuff. There wasn’t a single dull moment though – he’s hilarious and excited and smart.

He started talking about his recent work on hard-drive security. Did you know you can get a shell. On your hard-drive. A. Shell. These things don’t just have some firmware that optimizes reads and writes… they have a full shell where you can do things like repartition the disk, read and write data, and do other things.

How do you do this? Well, you hook up a serial cable and send the right commands. And it turns out, once you do that you can find out all sorts of crazy things the HD supports, and so, for example, he was able to find some hidden features for both Seagate and WD where he could use ‘hdparm’ to write malware to a hidden part of the drive that was immune to re-formattiong,
re-partitioning, etc.

His point here is that a computer is not in fact a computer – it’s a network of tons of tiny computers and many of them are vulnerable in unexpected ways. His other point was, if you’re doing security research or investigation… make sure there are no writable media when you are
executing malware. You can run it. Or you can write it. But never enable both happen.

He then talked about entropy. He got very angry about how we argue about theoretical vulnerabilities in PRNGs… when most software aren’t even using them, they’re using linear feedback shift registers which are total crap. His company has spent a lot of time getting “random()” in major languages to start using /dev/urandom instead of LFSRs and has been pretty successful.

He also talked about how browsers are fundamentally screwed because of they way they evolved – they are now essentially exposing internal COM objects to the outside world because of they way javascript is sorta layered on layout, and so it’s very hard to secure that. It turns out, after some research, that the vast, vast, vast majority of security issues with browsers are essentially use-after-free bugs because it’s very hard to know when some table was altered to have some reference to a thing which has a pointer to a thing that points to an image you thought you could free. IE and Chrome have both come up with solutions. One of them is grouping it’s memory types – so memory used to store an image will only ever be reused on other images. Memory used for tables will only ever be reused on other tables, etc. The other one is never ever re-using an address. It’s not that the memory for a whole image won’t be re-used it’s just that the virtual address will never get used again, which requires some work to give hints to the lookup table in the kernel, and requires a 64-bit machine to not run out of virtual memory addresses, but is very very secure.

Firefox, however, has no story. So Dan and a few other guys are spending the next few months getting a solution for Firefox. They started a mailing list, and they’re going to have a fork until they know they can perform as well as the existing allocator and then push it upstream (FF has a very fast internal memory allocator).

He also discussed a new system he’s built for passwords and fingerprints that uses a tuple of a noun, verb, and adjective.

It automatically fixes common typos, downcases, and canonicalizes all of the words (aeroplanes, airplanes, airplane all become ‘airplane’), and order of the three words doesn’t matter.

So your password would be:

crazy stampeding elephants

You could then type:

stampeeding crazy elephants

And it would work. The idea being that passwords become much easier for humans and much harder to crack (than an 8 character password with a bunch of crazy characters in it).

Code should be release shortly, as I recall.

I want to note here, he gave lots of credit to other people in- and out-side of his company for a variety of this, but I did not keep track of all of the various people who worked on various things.

My favorite quote, “Generation of random numbers is too important to be left to chance.”

Impostor — Polluting Tor Metadata

I felt bad for these guys. They had great content, but were just not good presenters. They weren’t comfortable in front of the audience and were barely audible and monotone. Unfortuantely, a lot of people left and missed what was actually a good presentation.

One of the problems with TOR is that people are starting to look for it. Not just the NSA, but IDSes have rules for it – Snort ships with several. So they wrote some javascript that you can just host that’ll cause all of your visitors to falsely trigger those alerts. They’re silly things like, it causes them to do a DNS lookup, or make an HTTP request to a TOR edge node. Not actual TOR traffic and nothing malicious, but enough that if people hosted this on some sites, there would be so many false-positives in IDS logs, it’d be useless to look for it.

The tricks they used to make these happen are very impressive. They even had lots of hilarious videos and jokes throughout their presentation. Sadly, they didn’t deliver these so much as just hit play.

Check Your Fingerprints: Cloning the Strong Set

This started by showing all of the many (well-known) failures of using PGP without checking fingerprints.

But they went on to something cool and new. They wrote a utility that will generate PGP keys with any keyID (the 32-bit keyID) you want. It does this by generating tons of keys on the GPU very very fast. The full fingerprint won’t be the same, but the keyid will which is all many people verify. Their method is so efficient, they were able to make their own version of every single key in the PGP strongset (same keyid, not same fingerprint, obviously) in a few hours. They also have a version that will create RSA keys very fast.

They also found a fun bug in GPG. If you request a key 0x12345678 from a keyserver, and it delivers a 0x88665544 (some other key), GPG will not warn you. It’ll show you they key it got and save it, but it doesn’t notice/care that its not the key it requested. And since public keys are… well, public, keyserver traffic is not secure. So, they can be MITM attacked. So they showed an attack where you do a gpg search, and a rogue keyserver on the local segment was able to see the request and send a rogue response. The client saw both responses and saved both keys. The rogue server sent one of their generated keys with the same name and keyID. They then asked gpg to verify the signature on a tarball of sourcecode that was signed with the ‘bad’ key, and of course it verified just fine.

This one is interesting – once might assume more than one valid key exists and simply make sure one of the two keys you have the right fingerprint. Which is fine, except that if you do the –verify with the short keyID, then you’ll pick the first one in the keyring. Since the rogue server answered first, the bad key was first.

So the second lesson is always use long keyIDs (58E11BB1E414D9AD vs E414D9AD).

The problem is that gpg by default doesn’t show this. So you either need --keyid-format long or to modify your ~/.gnupg/gpg.conf to include that.

Other Saturday Stuff

At night Kevin, Viv, their friend Scott and I went to dinner at Nove and then took in the Penn & Teller show from the VIP section! If you get a chance you really need to see them live, it’s incredible!

After that we headed to the DEFCON pool party, and ended the night chatting in a hotel room.

Sunday

NSA Playset: DIY WAGONBED Hardware Implant over I2C

These guys are seriously l33t. One of the NSA leaked docs showed they were using the I2C bus in computers to plant bugs in them. They intercept shipments, bug the I2C bus with a device with a GSM chip so you can extract data without even using the local network, and done. It’s supposedly practically undetectable because they send/receive all data not through your computer, so there’s no network anomalies. These guys set out to build that.

Turns out that the Video card has to talk I2C in order to be able to get resolution information from a monitor. So they decided to see if they could build a similar bug that would plug into a VGA port.

They got an arduino, a GSM board and a crypto board. I2C busses don’t cause hotplug events, so this is stealthy. The crypto board has a TPM chip on it, so they put the malware here in the TPM with keys in the TPM that can’t be extracted, so even if you find the malware, you can’t reverse engineer it. Smart.

They also use the crypto board to encrypt all traffic the bug sends or receives.

Because the bug has a GSM board and a SIM card they bought from t-mobile they can text it commands. It then drops off the command in an I2C slot at a known number (the demo used 42). Malware on the machine then polled that i2c mailbox and ran the commands on the target system. Similarly, the malware could ex-filtrate data by dropping it in the i2c mailbox and having the bug send it back to the ‘bad guys’ over GSM.

The CNC and ex-fil data never crosses your network, is encrypted with keys locked in a TPM.

Now this unit was several inches by several inches because it was built using arduino… obviously if you designed a board to do just this it could be VERY tiny and solder to a motherboard easily.

Scary stuff.

All the hardware schematics are open source and MOST of the code is open source. It’s been gutted in a few (documented) places to prevent script kiddie use, but should be easy to fix for security researches (or smart bad guys).

NSA Playset : GSM Sniffing

Another in the line of NSA playset movies. People have been working on easy sniffing of GSM traffic for the last year since the big talk on this last year.

Last year it required a $1000 software defined radio (SDR). These guys made MASSIVE progress. You can now buy a $20 USB SDR and grab all of the GSM data you want. GSM’s crypto is – as the talk showed last year – horrible, so they’ve fully automated the process of watching the dumps to find channels, grabbing the right packets, cracking the key, and decrypting all the traffic. Depending on traffic around you and the type of computer you have, you can probably decrypt all voice and SMS around you in real time with this.

DEFCON Kids

DEFCON kids does not allow adults that are not explicitly escorted by a child of theirs. In general it’s kids-only. But Viv is a teacher and scored a tour of this, so I skipped the 1pm tracks and took a tour of this. This is amazing!

They have kids in there learning to do packet dumps, pulling apart old hardware and soldering to make new devices, learning how to rf-shield things, learning how to do defense and pen-testing, lock-picking… and the whole thing is wrapped in a “these are super powers, with great power comes great responsibility” message. Google donated TONS of computers and engineer time to be in there teaching kids to hack as did several security firms. They also gave several chromebooks to the kids. I was particularly thrilled to see MANY girls in there.

Playing with Car Firmware or How to Brick your Car

This talk was hilarious. He hacked the entertainment system in his car and managed to get a shell and see it’s embedded QNX and even make some small changes. It turns out the system controls locks, A/C, lights, all sorts of things. And the features he didn’t buy were simply software-controlled and set to off. Unfortunately, 9 months into his research he uploaded a modified text file in which he believes he may have gotten the checksum wrong and bricked his car. It would still drive, but heat, AC, lights, etc. all didn’t work. It took 3 months to get it fixed… dealers how NO idea what was wrong or how to fix it. Then one that finally did told him “they replaced the car’s black box.” His wife won’t let him near the cars anymore with a laptop, but he’s published all his materials and how he figured out what he did and hopes others will continue his research.

Empowering Hackers to Create a Positive Impact

This talk was incredibly inspiring. Karen works with Masod & Israeli intelligence and has consulted with Cisco, IBM and countless other big companies. She’s… super smart.

Her talk was basically pointing out that the internet is ours to save, and that hackers ultimately are the “heroes” who bring about change, and we can make that change positive. So it was a call to arms to not keep bugs private, to do security research on the most important things like health devices and infrastructure, and practice responsible disclosure.

She argued that it’s easy to feel like “why should I tell my competition about this bug”… but a lot of the companies that are getting owned lately are not directly. RSA didn’t get hacked into, an executive at a company they used for building some component got phished. So the more people on the internet that are vulnerable, the more vulnerable you are. Because it could be a partner, or a vendor, or the CEO’s grandmother. So even if you don’t want to be altruistic about this, there’s incentive.

She also said Angelina Jolie’s character in Hackers was her inspiration as a girl to go into tech so she ended her talk by saying “hack the planet!!!”

Someone asked her how we can get more women in security, and she talked about finding mentors for them, among other things, but my favorite thing she said is “There are white-hats and black-hats… don’t be a douche-hat!”

Final Thoughts

I’ve been trying to go to DEFCON for years. For the last 3 years in a row I’ve had plane tickets and a hotel booked and had other stuff come up. Now that I finally made it, I plan to come every year I can. Next year I’ll add B-sides to the itinerary.

Running into Kevin and Viv and having people to share the awesome with definitely added an extra level of incredible to the weekend.

I also briefly ran into tons of other friends, acquantances and co-workers who all added to the experience. Thanks to everyone I saw this weekend!

-phil