A walkthrough to follow along

This current version is a deeper dive of how to exactly conduct an audit and I will walk through showing you the workflow and also importantly the entire workflow process of conducting an audit. Absolutely a practice use case.

Video Transcription

We've gone through the NIST cybersecurity framework and talked about some other frameworks. We've talked about compliance and regulations. And this is all fine. But really the work that relates to all of this, besides just implementing the program, is auditing, right? 

So you need to audit compliance standards, you need to audit, , like, like to prove that the controls are in place, right, like saying that you're doing firewall rules, doesn't mean that they've been implemented correctly, right? Like, like you can misconfigured devices, right? We see AWS s3 buckets all the time. misconfigured. Right. That's not because they stink at cybersecurity. And I'm sure in all those instances where an s3 bucket was breached, because it was open to the public. I bet you they were compliant, I bet you they had standards written, I bet you they had controls saying that we , we definitely secure our s3 buckets. And this is part of our configuration, change control process and releasing it and building like, you can have all that stuff, but crap happens, right? 

So auditing is the technique to check the efficacy of the controls themselves that you have in your environment, whether it's a compliance one, or you're actually doing it to measure the quality of your information security program, that's, that's oftentimes where I see it, right? If you're doing an enterprise risk assessment on an environment, you basically have to audit all the controls.

Now, I'm not going to go full into the lab. In this particular module. Look, I will be updating this module at a future time and replacing this video or adding to it. But what I want to talk to you about is how I would go about a lab. I had someone ask me this on stream the other day, and it's very, very pertinent to share this. Okay, so let me share my screen here. What can we do, I guess, let's do notes. 

Here, we do a new note. And this is a little janky. But , this is what we're doing, guys. And thank you for bearing with me. I appreciate it. Okay, so let's share the screen. Let's do a window. Let's make a new note. Okay, so we've got this new note here. This is a practical auditing. Now, let's just say I don't make the font bigger. I gotta make the font bigger, bigger. Can you guys see that? Is that too small for you guys? I want to make the font bigger but bigger. 

So let's say we're gonna audit the NIST cybersecurity framework, okay, like we're gonna go do we work for an insurance company or a big organization. And we're going to go audit them right or even better. NIST Special pub 800-171 Is the controls that are kind of underneath CMMC compliance, and they're the government's going to require contractors that do work with the government to to comply with those controls, right. So we're gonna say NIST SP 800-171. And I think it's something like 25 controls. 

So as far as audits go, there's several phases to the audit. And this is what the lab will be right? There is the audit prep, audit logistics. There's the actual audit itself on site. There's post, post audit, reconciliation, and there is analysis then there's reporting. Okay, these are the phases of executing an audit. 

It's a what are we doing an audit prep and audit prep, we're getting together all of our materials so we're gonna get the NIST 800-171 Excel sheet. Right? And, okay, so how can I share my screen real quick here Okay, let's take this risk.Alright, so we got this. You can see that they got some tools for you. They have a mapping to the Cybersecurity Framework, very valuable. Thank you very much. So let's do this. 

Let's look at 800-171. And this is a PDF so that's not going to help us too much. You could see the list of controls that are required so the So the requirements families, this is mapping to 800-53. I'm just kind of scrubbing right here to understand. Okay, here, this is what I would do if I was doing this NIST 800-171 file type xls. 

Okay, so now I don't even know what websites we're going to but I need an 800-171 Excel spreadsheet, because that's how I'm going to set my audit up. Okay. Alright, so I have to download the spreadsheet. Let's take a look at this one and see what it looks like. I know you guys are looking at that. Let's see, is this any good? Bear with me okay okay, um, okay, cool. So check this out. This is just the first result in Google that I pulled up, okay. Okay, and I know it's hard to see, but this is a NIST 800-171 in Excel. Get this a little tighter for you guys. Okay. And basically, it's a spreadsheet, guys. I know like people like that, that's kind of the banging against GRC is that it's just spreadsheets and stuff. But it is what it is. 

So what I would do is I would copy and paste this off, right? And we're getting. And you can see the NIST 800-53 control here. So you can go to the 800-53 and pull AC two and actually look at it. But we're saying that the control is basically that they're maintaining a list of authorized users, what they can access, what systems and all this stuff. And access requests are authorized before granted. So there's some workflow there's some process about new users coming on, how do they get access is like, can anyone request access to anything? Is there a form that needs to be filled out? Is there a workflow that a manager needs to approve? If someone's changing permission? Roles? Like, how is that access control? That's what this is all about. Right? 

Basically, you put this in now that I would add to this column, I would put basically a header on interview, or data collection type response. And this position, basically, and again, I'll do this all in the lab when I finally do it. But there's three ways to collect data, right? There's an interview here, let me switch to his interview. 

There's three ways to collect evidence, right guys, there's interviewing, there is document review. And then there's actual testing, right? Like trying to go to Pornhub on a work computer and seeing if the web filtering takes effect. Interviewing is what you think it is, it's where you sit down and you ask questions of the engineer or the analyst or the HR professional or the legal counsel on how they are doing these things? As part of when you when you , if there is a policy around access control, right, just to use this one as an example. What is there? Is there a form people fill out in order to request access? Let me see that form. Give it to me, that's part of this document review. Who reviewed how to show me the workflow processes that are documented so on how access is granted? Okay, good. Give that to me. Next, who, who is involved with this HR is involved with this, and there's an identity and access management team that's involved with this, okay? Like you basically just have to add them to your this is all part of your prep, right. Alright, so you've got all these, , which would be identified in the procedure document. Okay, so now you rinse and repeat guys, it's all rinse and repeat. 

Okay, so we move on to the next one. And if you don't know who to talk to, or how to validate this control, you can follow up with the NIST 800-53 to get more context and discussion and understand what's going on. So jumping back to our document here on how we're building this out. Okay, so we've got our spreadsheet now with what our scope of controls that we're going to be testing, and we've started to identify who we need to talk to and what we need to see Right. 

Okay, so now like you're, you're, you're kind of ready to go right, I'd also strongly encourage you. And this is like a best practice, you should read through all of the controls, especially the ones that you're going to be doing in an interview format, and make sure you understand exactly what you're going to be asking of those controls. I'm telling you right now I've seen it happen a million times, it's super embarrassing, it's super unproductive is when an auditor sits down, and they like literally read the control to the engineer, they're like, explain, like, explain how you comply with federal regulations that ensure that encryption standards adhere to minimum requirements of the practitioner. And then the engineer is gonna say, What the? What are you talking about? What like, what are you asking me, I don't get it. And then if you haven't done your work, if you don't know what you're talking about, all you're gonna do, and this is where it gets embarrassing, is reread the same thing you just read to them. And then the engineer is gonna say, I don't know what you're talking about. Now, you've lost credibility, because the engineer is thinking you're an idiot. And you don't know what you're doing. So you leave that interview with no clue on how to answer whether or not they've got the controls in place, right. 

So definitely do your prep work on what is going on. So understand what you're going to ask. Okay, now, let's go on to logistics. Guys, people have busy schedules, right? So who do you need to talk to? And like, can you bundle them together? Right? So if you need to talk to HR and IT about access control, right, you need to schedule that meeting, right? How long do you need? Is it a 30 minute meeting? Is it an hour-long meeting? 

You need to either do it yourself or have a liaison at the organization. If you're doing an internal audit, it's a little bit easier. But if you're external, you need a liaison point of contact to help logistically do this. Because before you show up on site, you need to have the schedule of events all lined up with who you're talking to, when you're talking to them, what we're talking about making them aware of what you're talking about in advance, providing additional context upfront. So when you get to the meeting, that 30 minute meeting, it is high impact, high value you're in and out. 

Trust me, people don't like auditors, okay? They don't want to waste their time talking to you. But it's important that you do the work. So you need to be prepared to execute and move and get it done. It's going to make them feel better about what you're doing. It's going to make them feel that you've prepared and that you're taking their time seriously. And , it's important. So make sure you schedule all those things upfront. 

Also, requesting SOPs, policies, documentation, right? Request all that stuff in advance, because you need to review it before you get on site. Right? So you're going to answer some controls already, like, yes, they have a policy, whatever. But if their policy in their SOP says that we do it this way, right? Every organization does things differently. We do it this way. Okay. So when you're in, when you're in the interview talking to them, you could say, Hey, I know your SOP says that you HR initially requests the user access, then it gets forwarded to the manager, then it goes to you and you actually execute on it. Is that how it's done? Yes or no? Yes. Okay. Well, let me see the last four new hires, and show me the spreadsheet or the workflow or the documentation or whatever, show it to me. And that'll validate that yes, you are, in fact, following the process. 

Having an SOP is easy. Anyone can pencil whip up an SOP actually following the process? That's a different story. Right? So if they say they do this for new hires, then show me the last four new hires. It should be there. And if it's not, then that's fine. You're just not following the process. So there's a problem in the process. But there's a problem with the way it's being pushed down by leadership on how to follow it. So you mark it up as not compliant, not in place, and you move on? Okay, so that's the on site. 

Okay. Again, with on site, high impact, high, , respect time. When you're actually doing on site audits, ask your questions, take your notes, if you can record the meeting. So you can go back if you want. If there's any follow ups, make sure because you'll get a lot of times where it's like, hey, how do you guys do this? And they're like, Oh, that's really more like Carl's thing. And Carl's out this week. We didn't know that Carl was supposed to be part of this interview, because you didn't ask us and it just didn't make sense like Carl never showed up. Okay, that's fine. I'll take an action here for me to follow up with Carl. 

So the week that you're on site, typically schedule the meetings, be early , knock it out. And then when you finished, possibly after each meeting, but at the worst at the end of the week, send a follow up email with outstanding data requests and action items you might have I have a follow up phone call, if you're, traveled to the site, but long story short, make sure that when you are finished with the on site piece of it, that you have all the data that you need in order to complete your work of the audit, because going back is not really going to be realistic. Okay? 

Okay, so now we've got our 800-171 audit, we've got all of our, we've got all of our data, our raw data on how they're doing these things. Now we're back in our office, okay? This is kind of where the post audit reconciliation is, you're, you're putting together artifacts supporting evidence, this policy maps to those control, right, it's like basically getting all your data tight, because when you were on site, you were just frantically taking notes and stuff like that you weren't trying to be clean about completing the work of the audit on site, you're literally just doing raw data gathering and analysis. So when you're doing your post audit, it's definitely about reading through the responses. Clearly understanding like this maps to the control, this is not exactly mapping to the control, there's no mission here, you yourself might actually discover that you overlooked something or skipped something. And I know I just said you really want to be careful about that. But it's important. Because you're going to have that follow up action email that you might like, if you can catch it, like on the Monday right after the audit, then you can kind of bundle it in there. But like if you come back like two weeks later, and you're like, oh, I need this. Like, it's just you could do it, but it looks unprofessional. Okay. 

Finally, analysis. This is where you actually go through and you say, is this In fact, in place or not? Right? Let me see. So here's our worksheet that we're going to be working from right. And, , they maintain a list of users and their roles, like, let's say that you find that they are maintaining the list, but they don't know who's got administrative access versus general user access, right. So they've got something in here, it's kind of a crappy process, whatever. You might mark your disposition states are not in place, not applicable. partially implemented, or fully implemented. Now with compliance, it's either compliant, or it's not compliant. But what I would argue is you want to document it, because even if the organization's doing something, they don't want to be marked as not in place, right? Like it's misrepresentative of what they're doing. So someone who does absolutely nothing and someone who's like 99% of the way, there are two different things, right. 

So , those are the states partially implemented, is a good one, you see it quite often fully implemented, not implemented, not applicable. These are the states. So you're going to go through each of these and look at your data. Think about what this control is asking. If you need to get additional information again, go to NIST 800-53. Look up the corresponding control, get context and then make the decision you're auditing. Is it? Are they complying or not? Okay, rinse and repeat. You'll hear that a lot in the compliance world, or audit world, rinse and repeat. 

So now you're going to go down, you're going to go through all these controls, and do all of this stuff. Okay, so let me go back to the notepad here. Okay, so we've done our analysis and made a disposition on each control. Okay. Now, we're at the reporting part, right, we've, so we're not doing risk work, by the way, guys, risk is a totally different thing. And you can take the outputs of this audit, and then do risk work. 

When we get to the risk section of the course, you'll see what I'm talking about, but you're not analyzing whether or not having this control partially implemented is a big deal. Like, let's say you have a control that's not implemented at all. How bad is that? Well, if you don't have multi factor authentication, that's terrible. If the control is , ensuring that you legally require third party risk managers to use levels of encryption that are acceptable stuff like that, like, like, yeah, that's important. But it's not nearly on the scale of multi factor authentication. So that this is where the risk work comes in, on kind of putting context to how crappy not having controls fully implemented is. 

But back to audit work. This is the report, right? So it was reporting, you're going to have an executive summary. This is basically like what happened in the audit, what is the disposition? Where are your areas of biggest gap, right? You could do like a pivot table and say like, you're not implemented in these areas like access control, you're pretty weak or business continuity, you're kind of weak, whatever. If you're doing an audit for the sake of regulatory compliance, you might just say, , you are compliant with the standard, you're not compliant with the standard, whatever you're going to want an executive summary, then you're going to want to include, like, the purpose of the audit. Scope of the audit, right? How did you do your audit? Like all facilities? Just one facility? Did you audit? Just one department, whatever? Who was interviewed, right? What else would you do? I guess, what you're auditing, like what you audited, right? So in this case, 800-171, for CMAC, compliance, and then results, right. 

So the executives are only going to read the summary, they're not going to dig into the details. All of this down here is for anyone that wants to fact check or validate the work that you did, right. So what I like to do here is either link to the spreadsheet, which is which is where all of my work was done that way, like, if anyone calls BS on anything that you've done, you should be able to trace back from the audit finding all the way through to your evidence to like what Carl said in an interview, and save this is how I came up with this disposition. Anything you put in this audit report must be objectively defensible, okay? Like you don't just kind of like blue, like just throw something in there, it has to be, like, remove yourself from the equation, another person should be able to pick up your work and come to the same conclusions that you came to it needs to be defensible, right, so I link to that spreadsheet. 

I would typically include, , maybe at a high level, compliance status. So showing where like, your access controls are fully compliant, or maybe like, the 25 controls, like just list them out, and then have next to it a column of like, control implementation status. It's very valuable. 

And then obviously, conclusion, okay. So, guys, this is, this is how you audit. This is, this is audit, one on one, you pick, you pick your framework, in this case, we are not your, your audit, standard, whatever you're auditing against, you get all those controllers, you put them in a spreadsheet, you get context over what all the elements are, you figure out who you need to talk to, for each of those. you schedule the meetings, yet you go on site, you execute, you get the documentation, you talk to the engineers, you request evidence to support processes and workflows that they said, you come back, you get all your data aggregated, put it together, you analyze whether or not the control is actually in place or not based on the evidence that you have, and then you report on it. It's as easy as that, guys. 

That's the demystification of audit work in 2022. All right. So that is it for this module. As I mentioned, my intention is to actually come up with a scenario and provide links to workbooks. And I'll have some fake documentation and stuff that we can use in order to practice conducting an audit. I just don't have the time right now. But it will be a course upgrade. 

That I'll be doing like if you have access to the course you'll get access to the upgrade as well. Alright, so let me check these comments really quickly. Yes. So hopefully, let me ask these questions as an auditor, would you provide a fix in order to help them ensure they get compliant? Yes. 

Good question, Nicolas. Personally, I do. Because , oftentimes, I have experience on what controls how it's been implemented in other organizations, effectively, I have a computer science background. So with the technical ones, I can kind of see that I built programs myself. So you can do that as a recommendation, you can include that actually, in the final report as recommendations for compliance. And they could take it or they could leave it. Definitely a nice to have, I would say you're, you're almost differentiating yourself as a value add auditor where they might want to come back to you a lot of times, honestly, guys, a lot of times, you're almost setting it up as follow on work, right? So like, oh, like, here's some recommendations. And oh, by the way, like my company also does this. So I know your firewalls need to be tuned or something like that. And my company also tunes firewalls. So if you'd like, here's a statement of work, and you can give us more money. So you do see that a lot of times there's always like, follow up work with all these things. 

Yep, so just so everyone knows, I have produced like 70% of the GRC course and I need to get this out by Monday. So what I'm doing here and you guys are experiencing it as I'm live streaming and just churning out the remaining 30% of this course . This is the content of the course right now. I need to get it published. And I will add updates to it as time allows. But I, for reasons that will become very obvious. When I reveal big stuff on Monday. I need to get this published before that date occurs for legal reasons. 

So let's see another question by Dylan. Any thoughts on how GRC relates to the world of outside directors? I don't understand that question. Till and can you rephrase it, please? Have you seen audits going full remote since the beer bug, thanks for the work on the course. The beer bug. I don't know what that means. But I mean, a lot of audit work is being done remotely. You can have an interview over teams, all these virtual collaboration platforms support all of this, you can request evidence and they can send it over. There's no reason why audit work can't be done remotely. I will say that I've been to the South Pole in Antarctica, for audit work. And that was one example where you really couldn't get there. You couldn't do it over the phone. Because they have internet down windows and stuff like that. But in most cases, yes, you can do it fully remotely. What is covering? 

What are you covering like CIP, would this work with CIP? Or would you act as a CI PP without the certification? What is CI PP for a second? I thought that that was like some type of privacy framework. Let me see. CiPP certification googling it right now, Kelly standby is a privacy standard. Yeah, I mean, anyone can audit, right? Like, as far as I know, there's no licensure that's required to audit if you want to get if you want to stand out, if you want to be a key differentiator, the i saqa C is a is the like official audit certification. So you could dig into that. But as far as CIP goes, like, 

Guys, you can audit anything. This is the methodology for auditing that I just, I just provided to you guys, whether you're auditing 800-171, or CIPP or whatever, you find out what the, what the comprehensive collection of things are, that the organization needs to be doing. Put it in an Excel spreadsheet, add columns for who you're talking to, what the response was, what the disposition of the control was, and if you want add a column for what evidence you collected, and basically figure out who you got to talk to then schedule meetings with them, then ask them the questions in the framework, then get the data and look at the controls and see if they're actually implementing them, then document all the control states or compliance right in place not in place, and then report on it. What do I mean? Like it's a methodology. It's not, this methodology will work with any, any audit, any standard, any framework anything, right? It's not, it's not restricted to certain styles. 

Okay. Okay, um, yeah, I mean, if you're going to get into audit CISA is definitely the one you want. No question. Alright guys, so this is thank you for being here. This has been the audit element of the GRC course of excuse me for the compliance and audit work module. We will just go into the final conclusion and touch points in the next section. So thanks