Why Collect Audit Logs?

Organizations collect audit logs for security and compliance purposes. With more services being run in the public cloud, teams are collecting security events from applications, systems, and cloud services running production services.

Audit logging is one of the many responsibilities that security team and DevOps team members must manage under the AWS cloud shared responsibility model. Collecting logs gives teams better visibility into activity that is happening in within their cloud infrastructure and organizations.  Audit logging standards are also required as part of regulatory compliance.

 

Auditing Compliance Requirements

Cybersecurity frameworks such as NIST 800-53, SOC 2, and ISO 27001 have specific requirements for collecting audit logs. Regulatory standards such as HIPAA/HITECH also have addressable audit logging standards, where organizations must collect logs.

Many security frameworks require that organizations have a process in place for collecting, analyzing, and resolving issues identified in logs. Logs are also useful for triage and emergency use. In the case of a potential security breach, organizations can view applicable logs to identify unauthorized access and see affected resources and data.

 

What to Collect in Amazon Web Services (AWS)?

With the many available cloud services, it can be difficult to determine how to configure audit logging for AWS. Below are types of logs organizations should consider collecting to aid in ensuring security, availability, and governance.

• AWS CloudTrails – Events related to AWS account activity and API calls
• AWS Access Logs – Events related to individual AWS access and cloud service queries
• System Logs – Events from operating systems and connected applications
• Availability Logs – Events detailing if cloud services and applications are running and available to end users
• Compliance & Security Configuration Logs – Events related to security settings and configuration.

In this article we will look at collecting logs using CloudWatch, S3 and other services. The services mentioned are very extendable and allow organizations to connect other services for further collection and analysis.

 

Enabling AWS CloudTrail

AWS CloudTrail is a service that allows cloud users to track user activity and API usage across the cloud environment. CloudTrail provides event history for AWS account activity, including actions taken through the AWS Management Console, AWS SDKs, command line tools, and other AWS services. Organizations can use AWS CloudTrail to get better visibility into user and resource activity in AWS and find malicious behavior by users.

Organizations should take the following steps to enable AWS CloudTrail and collect activity logs for your AWS Account or multiple accounts via AWS Organization:

 

Create a Trail for Your AWS Account:

1. 1. Sign-in to the AWS Account and open the CloudTrail console.
   2. Choose the AWS Region where you want the trail to be created.
   3. Choose “Get Started Now”. (Or choose “Trails”, if you do not see Get Started Now)
   4. On the Create Trail page, for Trail name, type a name for your trail.
   5. For Apply trail to all regions, choose Yes to receive log files from all Regions. This is the default and recommended setting. If you choose No, the trail logs files only from the Region in which you create the trail.
   6. For Management events, for Read/Write events, choose if you want your trail to log All, Read-only, Write-only, or None, and then choose Save. By default, trails log all management events.
   7. For Data events, you can specify logging data events for Amazon S3 buckets, for AWS Lambda functions, or both. By default, trails don’t log data events.

 

Create a Trail for Your AWS Organization

For teams that have configured AWS Organization with multiple AWS accounts, Trails may be configured across multiple accounts. Trail configuration for AWS Organizations can be found here. The basic steps can be found below:

1. 1. Sign-in to your AWS master account and navigate to the AWS CloudTrail console.
   2. In the region selector, choose the region where you want the trail to be created.
   3. Choose Trails, and then choose Create trail. (If you see Get Started Now instead of Trails, choose it.)

 

Enabling AWS Access Logs

AWS provides options for enabling the collection of access logs for specific cloud services. When enabled, AWS will collect and log cloud service queries. Logs are typically stored to S3 with processing being the responsibility of the user.

Organizations should enable access log collection for AWS services including the following:

 

Enable Server Access Logging for S3 Buckets:

1. Sign-in to your AWS master account and navigate to the AWS S3 console.

2. Select the Bucket to enable server access logging for.

3. Go to the “Properties” tab.

1. 
2. 4.Select “Enable Logging” and enter the target bucket (for where logs will be sent) and an optional target prefix (to have log object names start with the same name)

 

Enable Object-level Logging for S3 Buckets:

1. Under the “Properties” tab for S3 buckets, select “Object-level logging“

1. 

2. Select an existing CloudTrail trail in the same AWS region

3. Select whether to log “Read“, “Write” or both and click “Create“

 

Enable Logging for Other AWS Services:

Amazon S3 is just one cloud service to collect logs for. Organizations should look at collecting access logs and systems for other AWS cloud services and hosted database services. Security teams can reference the following guides for enabling logging with other popular AWS services:

Enable Access Logs for Elastic Load Balancers (ELBs)

Enable Logging for Amazon RDS

Enable Logging for Amazon Redshift

 

Collecting System Logs And Metrics

Organizations operating in the cloud generally have a number of virtual machines in Amazon EC2 or container-based services running in Amazon Fargate or ECS. These services rely on operating systems (OS) and software dependencies, that should be monitored for security issues, application errors and availability issues.

Amazon CloudWatch provides several different agents for connecting and sending operating systems and micro-service logs to CloudWatch. The CloudWatch Agent can be installed using the command line, AWS Systems Manager (SSM), or AWS CloudFormation. Checkout the following resources for installing the CloudWatch Agent:

Installing the CloudWatch Agent Using the Command Line

Installing the CloudWatch Agent Using AWS Systems Manager

Installing the CloudWatch Agent Using AWS CloudFormation

 

Monitor Availability With CloudWatch Logs

CloudWatch allows teams to view metrics related to availability, usage and service status. For example, teams can build and collect metrics such as CPU usage and read/write stats for services that are connected via the CloudWatch Agent. Teams can also setup metrics and create alarms to alert users of availability issues. Teams should consider utilizing CloudWatch logs and metrics to monitor overall availability of cloud services.

You can view a list of metrics collected by the CloudWatch Agent here.

Teams may consider creating CloudWatch Dashboards and CloudWatch Alarms for receiving notifications for critical availability events across the cloud environment. Organizations can configure CloudWatch alarms for specific AWS services and connect these alarms to notifications by creating an SNS topic.

 

Cloud Compliance Logs

Security teams must manage security configuration for many different AWS services and resources. As applications scale and new resources are created, DevOps team members must ensure that AWS resources have all necessary security settings. Organizations can utilize a solution like Dash Continuous Compliance Monitoring to detect and identify cloud security and compliance issues.

Organizations can detect compliance issues such as unencrypted volumes, open ports, disabled backup and access logging. Dash cloud security events logs may to other services for security analysis: