The Certification Industrial Complex

Don’t like reading? This is the blog version of the talk I gave recently at the Antisyphon snake oil summit. Watch it here. The Certification Industrial Complex I recently posed the question “What problem did certifications solve for you?” on a few different platforms. The overwhelming majority answered that it was to provide a proof of competence in some skill taught during the certification process. This “proof of competence” was generally to prove competence in a skill to themselves or to a potential employer (typically a new one). What I find interesting is that certifications are often presented as the only option for demonstrating this “proof of competence” and depending on the certification, it may not even prove competence in the advertised skills. We know that there are much better ways of demonstrating our knowledge, so why do we all turn to certifications when it’s time to learn something new? What can we do about it? To answer this, lets first get on the same page about what is meant by “proof of competence”. Proof of Competence: Verifiable evidence that someone has the ability to do something effectively. We as a security industry are terrible at both identifying (from the hiring perspective) and producing (from the job seeking perspective) this illusive proof of competence which has led to an over reliance on certification companies to act as an intermediary. Certification Pros While I’m about to spend a lot of time arguing that there are better ways to demonstrate your knowledge without relying on certifications, that doesn’t mean I am against them. I have a lot of certifications and will continue to get them if the situation is right and the opportunity cost of where I could spend my time/attention does not make it a waste. Additionally, certifications are probably still overwhelmingly the best option for most people looking to get their foot in the door in cybersecurity as they can somewhat-reliably provide something recognizable to get through HR filters. I do not wish to discredit the value these certifications can have in someone’s career nor do I think they should be looked down upon. I made a concerted effort early in my career to get certifications and I would not be where I am today without them. What I do wish to discuss is that they’re not the only option and there are better ways of both learning new material and demonstrating your mastery of it. As I’ve come to realize is the case with every contentious issue in security two (or more) ideas can hold true at the same time and certifications are not an exception. While this post mostly discusses the negatives aspects of certifications, it’s only because the positives are already so well documented. Additionally, this post is probably more relevant to those who already have a career in the field and are looking for ways to increase their skills after already establishing a solid understanding of the fundamentals. As I’ll discuss later, while certifications can very often lead to entry level roles, certifications typically aren’t enough to get you into more senior roles. Certification Overreliance If taking certification courses has taught me anything, it’s that there are multiple levels of understanding a topic. I mean, technically I have two project management certifications but I couldn’t tell you the first thing about SCRUM or AGILE other than they’re the butt of many jokes. This hierarchy of understanding is concisely summarized by Blooms Taxonomy which is a framework that allows for ranking someone’s knowledge of a subject and is typically represented by the following pyramid. If we apply the understanding of material tested by certification exams to blooms taxonomy, they really only teach at the bottom two tiers (remember/understand) and maybe the third tier (apply) if there is a good practical lab associated with the certification exam. For some concepts this is ok, however, if you want a higher level of competence, you must go further up the pyramid than the bottom tiers. This is where certifications fail, particularly for intermediate to advanced security concepts. Once you get past entry level positions, security jobs don’t only require knowledge of more material, they also require a deeper understanding of that material. Certifications struggle to address the deeper levels of understanding in a topic. Someone who passes $Certification exam can be an expert in that knowledge domain or be someone who just brain dumped the material to brute force the exam. The certification exam only displays a minimum level of competence in the subject and yet somehow certifications are the default tool we turn to when we need to demonstrate our competence in a subject even if we know they’re not great at it. Why is that? We’ve been pushed towards certifications for many valid reasons because they have many upsides. They’re great at getting your foot in the door, meeting compliance, and as a means to quell impostor syndrome, to name a few. The issue is that there comes a point where your time is not best spent chasing certifications but is instead best spent on other areas of learning but this is not often talked about. Certifications are a ruthlessly efficient way of profiting off of someone’s curiosity. The goal of certification companies is not to train you, it’s to generate profit and it is in the best interest of people who are trying to profit off of your curiosity to perpetuate the narrative that you can’t be competent in a subject unless you’re 🎉certified🎉 in it. This probably doesn’t help our terrible problem of impostor syndrome in the industry either. As it stands, technical competence is not the best determination for who succeed at a job and neither should it be. Technical competence is table stakes. It’s who can convince the employer they’re more competent that gets the job. You can be the most technically competent person in the world but if you don’t market yourself well to the company you’re applying for, you won’t get the job. Certifications struggle to prove even technical competence and they certainly don’t prove anything beyond it. A subtle distinction like this can lead to many qualified people losing jobs to less technically qualified candidates. Soft skills and personability aside, the best thing you can do to set yourself apart from others is to demonstrate your proof of competence in unique ways. These unique ways that I’ll discuss next provide more value than a simple skill check, they show persistence, technical competence, research aptitude, communication skills, writing proficiency, and possibly more. So how can you prove your competence having to rely on certifications? I believe there are at least two solutions to this problem but I would love to hear other ideas. The first is something that you can work on right now and the second is one that will take a cultural change in the industry. They’re both helpful on their own, but both should be worked toward for meaningful change to happen. A Path Forward One of the areas I’ve struggled the most with in my own career is spending time on training that isn’t tied to a certification because it’s difficult to learn something without having proof you’ve learned it. If what you learned helped you solve the problem you were having, why do you need further proof that you know what you’re talking about? We’re so tied up in proving that we’re competent in a subject that we forget that there are other avenues of learning that don’t end with a certification exam. If you’re a individual who is looking to demonstrate a your proof of competence in a skill, there are a few things that you can do that don’t rely on spending money for certification. Think of the following tips as more of a long term mindset shift that you should adopt, not a quick way to get a job. As flawed as they are, certifications are probably still the best way to get an interview if you need one quickly (even if they will only get your foot in the door) 1. Consider Opportunity Cost But Beware Of Getting Stuck On Tutorial Island Training can be expensive in the traditional sense of costing a lot of money but what is universally true is that it is EXPENSIVE on your attention. This is not by accident. Everyone wants you to spend your attention on their products that come in the form of certifications, courses, books, videos, and bootcamps. Opportunity cost must be at the forefront of your mind when deciding what you should be paying attention to. Spending 3 months studying for a certification means you likely won’t have time to learn other tools/technologies/concepts that you know are important. Ask yourself “Am I ok not being able to dedicate much time to other projects if I pursue this certification?”. You’ll find the answer is often “No…” followed by “…well I could just fit it in by doing…” Stop it. Don’t divide your attention too much or you risk not making any meaningful progress on anything. I speak from experience. I sometimes call this problem of jumping from one area to another being “stuck on tutorial island” and it’s one of the consequences of there being an unlimited demand for your attention from the smorgasbord of certifications/training/courses/videos and a limited supply of it. This can be for many reasons, including our own human nature, but mostly it’s due to highly effective marketing teams. Being stuck on tutorial island is a type of procrastination I fell into quite often before I realized what was happening and I see it all the time with people looking to learn new skills. In practice, this looks like getting halfway through something you’ve identified as important to learn, then switching to something new before fully learning the original concept. Personally this happened to me when I was getting into the field. I wasn’t sure if getting the CCNA or Network+ was better so I bought the CCNA material, got bored, switched to Network+, heard that CCNA looked better on resumes, restarted the CCNA material from the beginning, then finally switched back to Network+. This whole cycle took about 6 months of my time when it could have taken 1 or 2 months. In the end, it didn’t really matter which certification I got because the desired goal was always to be semi-competent in networking, NOT getting a specific certification. Instead, I wasted months of my time optimizing and second guessing what the best thing would be to have on a resume instead of just learning the material and moving on. You can read the first chapter of 10 text books or you could read all the chapters of one text book. The former is much easier to fall into even though it would still require the same amount of time and effort, but at the end of it you wouldn’t have any practical skills. 2. Create A Custom Learning Plan One underrated value of the training that comes with certifications is the curated list of concepts that you need to learn. This is helpful because it allows you to take a large concept and break it down into small tools/techniques/concepts that you can easily research on your own. The funny thing is, you’re not paying for the curated list of concepts, you’re paying for the training on those concepts which can range from 100% text based training all the way to in person instruction by an expert. There are pros and cons to all instruction methods. Whatever the instruction method, we often seem to forget that we’re not paying for the privilege to know the topics we are going to learn, we’re paying for someone to teach them to us. The #1 question I get asked is some variation of “What do I need to learn”. If you have the time, you can work backwards from where you want to be (IE: Pick a specific job role) and identify the skills you need to get that job. Then you can create your own custom learning plan to teach you those skills. Approaching it this way is not nearly as easy as clicking “Add to cart” on a certification, but you’ll not only save money, you’ll also gain a better understanding of the material that you’re actually interested in by putting together a learning plan that is specifically tailored to your own goals. Fortunately for those looking to get into this field, our field built upon open source technology and techniques which means there is very little information in our field that cannot be found online for free. There are, of course, exceptions but for the vast majority of subjects, this is true. This makes creating your own custom learning plan even more powerful, especially when most certifications are just teaching open source tools and techniques that are forged in public discussions, blogs, and git repositories. Lets look at an example of how I would approach creating my own custom learning plan. The following syllabus is from the course outline I picked at random, but you could do this with any course. Distillation: Pull out all the concepts you wish to learn or don’t understand from a variety of places. Read reviews of the certification, read the syllabus, etc. Prioritize: Using the list you created from the last step, create a list and rank how well you wish to understand something Research: Find find resources that will allow you to learn each item on your learning list. Training, books, videos, blogs, etc. Execution: Put the time in to learn the material. Below is an example learning plan of the above course. I’ve mapped each concept to Blooms Taxonomy so that I can identify what level I wish to learn the topic on. I’ve also given it a time commitment and “bandwidth” column that represent how long it’ll take me to learn and how difficult it’ll be. The rationale being that concepts that are highly complex should be given the proper time and mental bandwidth to learn whereas “easy” things can be learned in tandem with other concepts. Note that this is a quick example and your learning plan should probably be a bit more thorough. Additionally, you should add to your learning plan as you come across new concepts that you wish to learn. This will allow you to tailor what you learn to your goals. Spoiler alert, if you’re doing it right, you’ll never “complete” everything on your list because the more you learn the more you’ll have questions on. Maybe there are 3 certifications on penetration testing that all list different concepts in the syllabus. Creating your own custom learning plan will allow you to add in concepts from multiple sources to your learning plan so your knowledge is even more comprehensive. Is this more work than simply buying a certification and taking the exam? Absolutely. Is it worth it? I would argue so as the outline I highlighted above is being sold for over $8,000. If you are looking to demonstrate to others what you have learned, there are many methods you can use to demonstrate it which I will discuss shortly. 3. Ignore the outputs, prioritize the inputs If the end goal is to prove your competence, the best thing you can do is actually be competent. By focusing on becoming as much of an expert as you can on a topic, you eliminate the need for an external party to certify that you know what you’re talking about. Certification companies have great marketing teams that have done wonders to inflate the importance of having their certification, not the importance of being competent in a skill. Don’t worry about needing a certification to prove that you have a skill, by focusing on actually learning that skill, you will gain a much deeper understanding of it and it will be apparent to anyone you talk to about the subject. This means focus solely on the training aspect of learning and pay little attention to the certification aspect. Admittedly this is a slower approach than only studying the material that will allow you to pass a certification exam, but it will result in a more well rounded understanding of the subject area. If you still want to take the certification exam at the end of your studies, by all means do so. I’d urge you to be more interested in the learning itself rather than the certification though. 4. Create Your Own Proof Of Competence Once you’ve honed your knowledge in a skill, you will be able to create your own proof of competence that doesn’t rely on a certification and should be valued more than a certification (I say “should” because that takes an interviewer to actually know what they’re looking for, which I’ll discuss more shortly). I’ve put together some ideas of ways you can demonstrate your competence to both yourself and others. This is a short list of ideas I came up with but I would love to hear any other ideas. Doing new research in the subject and writing about it. Creating a new tool around the subject. Contributing to existing open source tooling on the subject. Writing a blog post synthesizing your research and knowledge. Creating videos explaining the subject. Doing a writeup of resources you used to learn about the topic. Speaking at a conference about the topic. Building it in your home lab and documenting it. You’ll notice that most of these are publicly demonstrating your competence in an area. This is important a few different ways. First, it eliminates the need for an external third party to validate your competence in a skill which is what certifications aim to accomplish. Assuming you’re actually producing a quality blog/talk/video on a topic, anyone who is looking at your credentials can easily see if you actually understand what you’re talking about. It will take a little more time and effort from those looking to hire you than a simple certification, but it can more be a more accurate barometer for someone’s skill level than a simple pass/fail certification exam. Additionally, if a potential employer sees that you have a ton of research, open source contributions, and conference talks on a topic and they don’t choose to hire you over someone who only has a traditional certification in that area, you probably are better off not working there in the long run anyway. Second, publicly demonstrating proof of competence in an area increases your connection to the community. The security community is tiny. Putting out new research, creating tools, speaking at conferences, etc make you stand out in the community much more than simply having a certification in your bio. All of these demonstrate that you’re actually competent in a subject and have the added benefit of being great things to talk about in interviews. Nearly every interview I’ve had has brought up conference talks I’ve done and blogs I’ve posted. I’ve never been asked about my certifications in an interview. The caveat is that you actually need to put time and effort into these projects. Putting out low quality work can have the opposite of the desired effect. A Collective Path Forward I’ve touched on how you as an individual can decrease your need for third party certifications, but that is only half the equation. The other half is from the hiring side and it is a bit more difficult to circumvent. Unfortunately, certifications allow for companies looking to hire to be lazy. We’ve shifted to a model of outsourcing the vetting of employee’s technical skills to third party certification companies who honestly aren’t even doing a very good job. Certifications offer a quick way of filtering resumes and identify who has an basic understanding of a required skill, but they lack the depth required to make a determination into someone’s level of competence. The remedy to this is fairly simple on paper but difficult in practice: Those in charge of hiring need to celebrate higher quality proofs of competence over traditional certifications. This is something that we probably already know in the back of our mind, but it’s almost never put into practice. I have an idea of why but that’s a rant for a different time. This is a difficult thing to change because it requires a cultural shift, it’s not an AI product that can be bought or a KPI that can be easily measured. Fortunately, I believe there really is a return on investment on hiring people who have demonstrated higher level proofs of competence even if it’s difficult to quantify. Someone who has put the time and effort into researching a new area of a subject, speaking about it at conferences, or synthesized information into a blogpost, is very likely much more qualified for a position in that area than someone who has a bunch of fancy 4 letter certifications behind their name (And I’m speaking as someone who has 17 certifications…). I think we all intuitively know this, it’s just difficult to quantify. If you’re in a position to hire someone, celebrate higher quality proofs of competence over traditional certifications and you’ll find you’re hiring process easier and turnover rate lower. Other Cyber Education Problems While working on this post, I identified that there are LOTS of other issues in the area of cybersecurity education. In the rest of this post I aim to identify issues I see, even if I don’t necessarily have solutions for them yet, I think they’re worth talking about. If you have some solutions to the following, I’d love to hear about them. You can reach out to me here and let me hear what you’ve got! Bootcamps Bootcamps get a bad rap from most people in the industry for good reason. While there can be amazing benefits to them, those benefits aren’t mutually exclusive to bootcamps, they’re really from learning. Most of which you can learn for free or very cheap from many places. The majority of security bootcamps I’ve come across are a thinly veiled appeals to wealth. Bootcamps also offer the worst of both worlds, they’re typically insanely expensive and they have almost no recognition from other companies. Can they work? Probably, but do they work better than spending the same amount of time and effort learning on your own? You need to seriously vet any type of boot camp you take, there are lots of bootcamps out there that give you the bare minimum education that you could get from any free training course and maybe a little bit of career advice. Career advice is fine most of the time (as I’ll discuss next), but career advice isn’t a bootcamp just as much as telling students to take certifications is not a bootcamp. The issue is that people who are new to the field don’t know how to properly vet a bootcamp which leads to a LOT of predatory pricing on bootcamps. If they could vet the bootcamp, they probably wouldn’t need a bootcamp because they can research something on their own which means they can learn on their own. There is lots of information out there on how to get into the field. Research how others got into the field and find the commonalities between them. The core skills you find that come up over and over are probably what you should focus on learning. You don’t need a bootcamp to tell you that. The Perils Of General Advice There are many people who gain a large following by simply parroting generic advice, unfortunately, bad advice can be worse than no advice as it can lead you to waste your time and money. There are many people who are very eager to give out career advice on how to get a job in security without ever having working in a security job. I don’t know anything about accounting so it wouldn’t make sense for me to give career advice or recommendations about the pros and cons of becoming a CPA even if I know a lot of CPAs. Those who are new to the industry can easily be led astray by those with a large following who don’t quite understand what they’re advising people to invest their time into. Unfortunately, giving generic advice or recommendations is a great way to boost engagement on most platforms which only further incentives the cycle of low quality, generic parroting of other people’s advice. When you’re not qualified to give advice on a topic and you do so anyway, you can greatly misguide your audience and misinformation can be far worse than no information. This is particularly pronounced when advice on certifications are tossed into the mix. Giving advice on if someone should take a certification without having gone through the material yourself, understanding that person’s goals and motivations, and having experience in the field is a recipe for disaster. There are a few things that can help you identify the snake oil: Are they constantly putting other people down? Does the person giving advice have a background in this area? Are they benefiting from selling you something? Are they engagement farming? (“Comment below your top 3 certifications!!! 👇👇👇”) Does the person share resources on post insightful information or copy/paste quotes from articles? Does the person charge a fee to ask them questions? Have they contributed to the community (besides “curating” other peoples content). Note that these are general guidelines I use to suss out if something seems fishy, just because someone matches one of these criteria doesn’t mean they’re selling snake oil. Not to mention the shady deals that some companies make behind the scenes where influences are paid to promote things that aren’t always disclosed properly but we’ll just not talk about that… Fin? If I’m being honest, this was not a fun post to write, but I feel that these are some of the insider baseball lessons I’ve learned over the past few years of being in the industry that no one talks about. If you’re interested, I gave a talk about this recently at the Antisyphon snake oil summit. If you have any questions, comments, typos, or anything else, feel free to reach out to me on any of these platforms.

Don’t like reading? This is the blog version of the talk I gave recently at the Antisyphon snake oil summit. Watch it here.

Leave a Comment