Lab 05 Linux Authentication & System Log Investigation

Mission Overview

Welcome to Lab 05 — Linux Authentication & System Log Investigation.

In Lab 04, you investigated Windows events and correlated authentication, privilege, process, service, and change-management evidence.

Now you will apply the same disciplined SOC methodology to Linux systems.

Linux investigations frequently rely on multiple data sources rather than one centralized event channel. Depending on the distribution and configuration, evidence may appear in:

• authentication logs

• system journals

• SSH logs

• sudo events

• service logs

• account-management records

• application logs

• endpoint telemetry

• network telemetry

Your goal is not to memorize every Linux log path.

Your goal is to determine:

What happened, which identity performed the activity, from where, under what privilege level, what services or processes were involved, and whether the evidence supports legitimate administration, an operational issue, or suspicious host activity.

Mission Goal: Investigate Linux authentication and system activity by correlating SSH, sudo, account, service, process, system-journal, and network evidence into a defensible host timeline and determine the appropriate SOC disposition.

Mission Information

Item	Details
Difficulty	Intermediate
Estimated Time	120–150 minutes
Primary Skill	Linux Log Investigation
Secondary Skill	Authentication & Privilege Analysis
Environment	GoHackersCloud SOC Analyst Lab
Input	Linux logs + SIEM/endpoint/network evidence
Primary Outcome	Linux Host Investigation Report
Safety Level	Authorized Training Evidence Only

Learning Objectives

By completing this lab, you will be able to:

• identify common Linux security log sources

• investigate SSH authentication

• analyze failed and successful logins

• identify source systems

• review sudo activity

• distinguish sudo use from privilege escalation

• analyze user switching

• review account creation and modification

• investigate service activity

• correlate process evidence where available

• review system-journal evidence

• identify operational authentication problems

• investigate service-account behavior

• correlate Linux events with network telemetry

• reconstruct a Linux incident timeline

• identify telemetry limitations

• determine host and user scope

• assign severity and confidence

• create escalation recommendations

• produce a professional Linux investigation report

Investigation Methodology

Use:

Linux Event → User → Source → Process/Service → Privilege → Time → Correlation → Conclusion

Conceptually:

Linux Alert / Event
        │
        ▼
     Identify Host
        │
        ▼
     Identify User
        │
        ├── SSH
        ├── sudo
        ├── su
        ├── Service Account
        └── Local Activity
        │
        ▼
   Process / Service
        │
        ▼
 Privilege Context
        │
        ▼
  Related Log Events
        │
        ▼
 Endpoint + Network
        │
        ▼
      Timeline
        │
        ▼
      Scope
        │
        ▼
   SOC Disposition

The core principle is:

Linux authentication or privilege activity is not automatically malicious. Context determines security relevance.

Part 1 — Create the Investigation Workspace

Create:

SOC-Labs/
└── Lab-05/
    ├── 01-Scope/
    ├── 02-Alert/
    ├── 03-Authentication/
    │   ├── SSH/
    │   ├── Local-Login/
    │   └── Service-Accounts/
    ├── 04-Privilege/
    │   ├── Sudo/
    │   └── User-Switching/
    ├── 05-Accounts/
    ├── 06-Processes/
    ├── 07-Services/
    ├── 08-System-Journal/
    ├── 09-Network-Correlation/
    ├── 10-Timeline/
    ├── 11-Evidence/
    ├── 12-Findings/
    └── 13-Report/

Create:

Lab-05-Linux-Authentication-System-Log-Investigation.md

Part 2 — Open the Case

Example:

CASE ID:
GHC-SOC-2026-005

CASE TITLE:
Linux Authentication & System Log Investigation

HOST:
LINUX01

STATUS:
Investigating

INITIAL PRIORITY:
P2

ANALYST:

DATE:

Part 3 — Review the Initial Alert

Assume the lab provides:

ALERT ID:
ALT-LNX-001

ALERT:
Suspicious SSH Authentication Followed by Privileged Activity

HOST:
LINUX01

USER:
lab-user

SOURCE:
10.10.20.55

TIME:
15:20 UTC

INITIAL SEVERITY:
High

Do not conclude:

Linux server compromised.

Start with:

What happened on LINUX01 before, during, and after 15:20 UTC?

Part 4 — Define the Investigation Window

Start with a limited window.

Example:

Alert Time:
15:20 UTC

Initial Window:
15:05–15:40 UTC

Expand when the evidence justifies it.

Part 5 — Build the Linux Host Profile

Record:

LINUX HOST PROFILE

Hostname:

IP:

Distribution:

Kernel:

Business Role:

Criticality:

Owner:

Expected Administrators:

Expected Services:

Normal Administrative Sources:

Endpoint Monitoring:
Yes / No / Unknown

Part 6 — Identify Available Linux Log Sources

Depending on distribution, useful sources may include:

/var/log/auth.log
/var/log/secure
/var/log/syslog
/var/log/messages

Systemd systems commonly provide:

journalctl

Do not assume every Linux distribution uses the same files.

Part 7 — Record Log Availability

Create:

Log Source	Available	Purpose
auth.log / secure		Authentication
system journal		System/service activity
syslog/messages		General system events
application logs		Application context
endpoint telemetry		Process/file/network

Part 8 — Review Authentication Logs

Where applicable:

sudo tail /var/log/auth.log

or:

sudo tail /var/log/secure

For a time-limited journal review:

journalctl --since "2026-08-28 15:05:00" --until "2026-08-28 15:40:00"

Use read-only investigation commands.

Part 9 — Investigate Failed SSH Authentication

Look for evidence such as:

Failed password for lab-user from 10.10.20.55

or:

authentication failure

Record:

FAILED SSH AUTHENTICATION

Time:

User:

Source IP:

Target:

Authentication Method:

Failure Reason:

Count:

Expected Source:
Yes / No / Unknown

Evidence:

Part 10 — Investigate Successful SSH Authentication

Look for entries such as:

Accepted password

or:

Accepted publickey

Record:

SUCCESSFUL SSH AUTHENTICATION

Time:

User:

Source:

Authentication Method:

Session:

Expected User:
Yes / No

Expected Source:
Yes / No

Expected Time:
Yes / No

Part 11 — Build the Authentication Sequence

Example:

15:12
Failed SSH login — lab-user — 10.10.20.55

15:13
Failed SSH login

15:14
Failed SSH login

15:17
Successful SSH login

15:18
Session opened

15:20
sudo activity

The sequence is more meaningful than individual lines.

Part 12 — Determine the Authentication Method

Common SSH authentication methods include:

• password

• public key

• certificate-based authentication

• keyboard-interactive

• MFA-integrated authentication

Record:

Authentication Method:

Expected for User:

MFA:
Yes / No / Unknown

Security Significance:

Part 13 — Interpret Repeated Failures Carefully

Possible explanations include:

• mistyped password

• stale credential

• automation

• service account issue

• password guessing

• unauthorized access attempt

Remember:

SSH Failures
    ≠
Brute Force Automatically

Part 14 — Search the Same Source Against Other Users

Create:

User	Failures	Successes	Source
lab-user			10.10.20.55
admin
svc-app

If many users receive a small number of failures from one source, investigate for a broader authentication pattern.

Part 15 — Search the Same User Across Other Sources

Create:

Source	Failures	Success	Expected
10.10.20.55
10.10.30.12

This may reveal:

• multiple devices

• jump hosts

• VPN behavior

• shared-account misuse

• unexpected access

Part 16 — Build the Source Authentication Profile

SOURCE AUTHENTICATION PROFILE

Source IP:

Known Asset:

Hostname:

Zone:

Owner:

Users Targeted:

Failures:

Successful Sessions:

Privileged Accounts:

Expected Administration Source:

Assessment:

Part 17 — Review Session Open and Close Activity

Authentication logs may show:

session opened
session closed

These help establish:

• session duration

• active user

• privilege transitions

• timing

Build:

User	Opened	Closed	Duration

Part 18 — Investigate sudo Activity

Linux administrators commonly use sudo.

Log evidence may indicate:

user : TTY=pts/0 ; PWD=/home/user ; USER=root ; COMMAND=...

Record:

SUDO EVENT

Time:

Invoking User:

Target User:

Command:

TTY / Session:

Working Directory:

Expected:
Yes / No / Unknown

Business Context:

Part 19 — Understand sudo Correctly

Remember:

sudo Used
   ≠
Privilege Escalation Vulnerability

If an account is authorized to use sudo, privilege elevation may be expected administration.

The questions are:

• Is the user allowed?

• Is the command expected?

• Did it occur from the expected source?

• Was it within an approved activity window?

Part 20 — Build the Sudo Activity Register

Time	User	Target User	Command	Expected
		root

Part 21 — Review User Switching

Linux users may switch identity using mechanisms such as su.

Record:

USER SWITCH EVENT

Original User:

Target User:

Time:

Source Session:

Expected:

Evidence:

Again:

User Switch
    ≠
Account Compromise

Part 22 — Build the Privilege Timeline

Example:

15:17
lab-user authenticated

15:18
session opened

15:20
lab-user invoked sudo

15:21
privileged command executed

15:25
session closed

Then ask:

Does this match the user’s expected administrative role?

Part 23 — Review sudo Authorization

Where the lab supplies configuration evidence, determine whether:

lab-user

is expected to have sudo rights.

Do not modify sudo configuration.

Record:

SUDO AUTHORIZATION

User:

Authorized:
Yes / No / Unknown

Scope of Privilege:

Expected Administrative Role:

Assessment:

Part 24 — Investigate Account Creation

Linux account-related events may appear through:

• auth logs

• system journal

• administrative tooling logs

• account files/endpoint telemetry supplied by lab

Record:

ACCOUNT CREATION

Time:

New User:

Creating User:

UID:

Group:

Shell:

Home Directory:

Expected:

Change Reference:

Part 25 — Investigate Account Modification

Review evidence involving:

• password changes

• account locking/unlocking

• shell changes

• group membership changes

• privilege changes

Do not treat every account change as malicious.

Part 26 — Build the Account Activity Register

Time	Actor	Account	Change	Expected

Part 27 — Review Privileged Group Membership

Important Linux groups may vary by distribution and environment.

Examples may include administrative groups used for sudo authorization.

Record:

GROUP CHANGE

User:

Group:

Actor:

Time:

Expected:

Security Relevance:

Part 28 — Review Process Evidence

Where endpoint telemetry or process-accounting evidence is provided, record:

• process name

• parent process

• user

• command line

• start time

• network activity

Create:

Time	User	Process	Parent	Assessment

Part 29 — Avoid Process-Name Assumptions

Examples:

curl
wget
python
bash
ssh
scp

can all be legitimate administrative tools.

Therefore:

Tool Present
    ≠
Malicious Activity

Investigate the context.

Part 30 — Build the Process Context Profile

PROCESS:

User:

Parent:

Command:

Working Directory:

Network Connection:

Expected Role:

Associated Change:

Assessment:

Part 31 — Investigate Shell Activity Carefully

Shell execution is normal on Linux servers.

A process such as:

bash

does not independently indicate malicious activity.

Look for:

• who launched it

• from which session

• what evidence exists around it

• whether the user normally administers the host

Part 32 — Review Service Activity

Linux services may be controlled through systemd or other init systems.

Use read-only review where permitted:

systemctl status <known-service>

or journal review:

journalctl -u <known-service>

Do not start, stop, or reconfigure services.

Part 33 — Build the Service Activity Register

Time	Service	Event	Actor	Expected
		Start/Stop/Restart

Part 34 — Investigate Unexpected Service Changes

Possible reasons include:

• software upgrade

• administrator restart

• application failure

• deployment

• suspicious persistence

Correlate with:

• user

• sudo

• process activity

• change record

• endpoint evidence

Part 35 — Review System Journal

Useful read-only approaches include:

journalctl --since "15:05" --until "15:40"

or service-specific filtering.

Look for:

• authentication

• service changes

• kernel/system messages

• application issues

• scheduled activity

Do not treat routine system warnings as security incidents automatically.

Part 36 — Investigate Scheduled Activity

Linux may use:

• cron

• systemd timers

• application schedulers

If the lab provides evidence of scheduled execution, determine:

Who configured it?

What executed?

When?

Is it expected?

Do not modify scheduled tasks.

Part 37 — Build the Scheduled Activity Register

SCHEDULED ACTIVITY

Time:

User:

Scheduler:

Command / Service:

Frequency:

Expected:

Change Record:

Assessment:

Part 38 — Investigate Service Accounts

Service accounts often produce activity very different from human users.

Ask:

Should this account log in interactively?

Which host normally uses it?

Which service owns it?

Does it require shell access?

Why is it authenticating now?

Part 39 — Example Suspicious Service Account Pattern

Suppose:

svc-backup

normally authenticates from:

BACKUP01

but you observe:

svc-backup authentication from USER-PC-17

That deserves investigation.

Do not assume compromise without additional evidence.

Part 40 — Build the Service Account Baseline

Account	Expected Source	Interactive Login	Privileged	Observed
svc-backup	BACKUP01	No	Limited

Part 41 — Review Network Correlation

After suspicious Linux authentication or privileged activity, correlate:

• firewall

• DNS

• proxy

• endpoint network telemetry

Look for:

Linux Host
    ↓
Unexpected Destination

or:

Source Host
    ↓
SSH
    ↓
LINUX01

Part 42 — Build the Network Correlation Register

Time	Source	Destination	Port/Protocol	Expected

Part 43 — Correlate DNS Activity

Where DNS telemetry exists, assess whether unusual processes or sessions produced related queries.

Record:

DNS CORRELATION

Time:

Client:

Domain:

Process:
If available

Expected:

Security Relevance:

Part 44 — Correlate Firewall Evidence

Firewall evidence can help validate:

• incoming SSH source

• outbound connections

• port/protocol

• allowed/denied status

Remember:

Firewall Allow
    ≠
Malicious Activity

It only confirms a network path.

Part 45 — Review Change Management

Administrative Linux activity may be completely legitimate.

Look for:

• maintenance window

• deployment

• patching

• backup activity

• service restart ticket

• administrator assignment

Create:

Event	Change Reference	Match	Assessment

Part 46 — Build the Host Correlation Matrix

Time	Authentication	Privilege	Process	Service	Network
15:17	SSH success
15:20		sudo
15:21			process
15:22				service restart
15:23					external connection

This allows the analyst to see the story.

Part 47 — Build the Master Linux Timeline

Example:

15:11
First failed SSH login

15:13
Additional login failures

15:17
Successful SSH authentication

15:18
Session opened

15:20
sudo command executed

15:21
Service configuration activity observed

15:22
Service restarted

15:24
Outbound connection observed

15:30
SSH session closed

Populate only evidence that exists in your lab.

Part 48 — Build Investigation Hypotheses

Example:

Hypothesis 1 — Legitimate Administration

Expected admin source
+
authorized user
+
maintenance window
+
expected service action

Hypothesis 2 — User Error Followed by Normal Login

same expected source
+
few failures
+
successful login
+
normal activity

Hypothesis 3 — Unauthorized SSH Access

unexpected source
+
successful login
+
unusual sudo activity
+
no maintenance record

Hypothesis 4 — Service Misconfiguration

service account failures
+
fixed interval
+
known recent password change

Part 49 — Create the Hypothesis Matrix

Hypothesis	Supporting Evidence	Contradicting Evidence	Confidence
Legitimate admin
User error
Unauthorized access
Service issue

Part 50 — Determine Host Scope

Search for the relevant:

• account

• source

• process

• service

• indicator

across other Linux hosts where telemetry is supplied.

Create:

Host	Related Event	Same User	Same Source	Status
LINUX01	Yes	Yes	Yes	Investigating
LINUX02

Part 51 — Determine User Scope

Create:

USER SCOPE

Primary User:

Other Users Targeted:

Privileged Users:

Service Accounts:

Accounts with Successful Sessions:

Accounts with Failed Attempts Only:

Part 52 — Determine Privilege Scope

Ask:

Was sudo used?

Was root access obtained?

Was privileged access expected?

Which commands were documented?

Were administrative groups modified?

Important:

Do not claim privilege escalation merely because root-context commands occurred through authorized sudo.

Part 53 — Determine Service Scope

Create:

SERVICE SCOPE

Affected Service:

Host:

Unexpected Changes:

Restarted:
Yes / No

Configuration Change:
Confirmed / Suspected / Not Observed

Operational Impact:

Security Relevance:

Part 54 — Identify Telemetry Gaps

Examples:

• no command-line telemetry

• authentication logs rotated

• source IP unavailable

• no endpoint agent

• no process accounting

• missing sudo logs

• no DNS telemetry

Record each gap.

Part 55 — Build the Telemetry Gap Register

Gap	Impact	Confidence Effect	Follow-Up
No process telemetry	Cannot confirm command execution	Medium	Endpoint review
Missing source	Attribution limited	High	Network logs

Part 56 — Identify Positive Security Controls

Examples:

SSH access restricted to management network.

Public-key authentication enforced.

MFA required for remote administration.

sudo activity logged.

Service account interactive login denied.

Failed authentication generated SIEM alert.

Outbound traffic restricted appropriately.

Part 57 — Build the Linux Security Control Matrix

Control	Status	Evidence
SSH logging
MFA
Source restriction
sudo logging
Service-account restriction
Endpoint telemetry
Network monitoring

Part 58 — Determine the Disposition

Use:

Expected Administrative Activity

Activity matches approved administration.

Benign User Activity

No relevant security concern.

Operational Issue

Example:

• stale service credential

• broken automation

• failed deployment

Suspicious Host Activity

Unusual behavior requires further analysis.

Potential Security Incident

Multiple correlated indicators support possible unauthorized access.

Confirmed Security Incident

Evidence clearly demonstrates unauthorized or malicious activity.

Inconclusive

Evidence is insufficient.

Part 59 — Example — Legitimate Administration

DISPOSITION:
Expected Administrative Activity

HOST:
LINUX01

USER:
admin01

EVIDENCE:
SSH authentication originated from the approved management
workstation during the scheduled maintenance period.

The user is an authorized Linux administrator.

Subsequent sudo activity and service restart align with
CHG-2026-305.

No unrelated suspicious activity was observed.

SEVERITY:
Informational

CONFIDENCE:
High

Part 60 — Example — Operational Service Account Issue

DISPOSITION:
Operational Authentication Issue

ACCOUNT:
svc-backup

EVIDENCE:
Authentication failures occurred every five minutes from the
approved backup server.

The service account credential was changed earlier in the day.

No successful authentication from unexpected systems was observed.

ASSESSMENT:
Evidence is consistent with the backup service retaining an old
credential.

SECURITY IMPACT:
Account lockout risk and monitoring noise.

ESCALATION:
Backup application owner.

Part 61 — Example — Suspicious SSH Activity

DISPOSITION:
Potential Security Incident

HOST:
LINUX01

USER:
lab-user

SOURCE:
Unrecognized internal workstation

EVIDENCE:
Successful SSH authentication occurred from a system not included
in the user's normal administrative baseline.

The same session was followed by unexpected sudo activity.

No approved maintenance activity was identified.

KNOWN IMPACT:
No destructive activity or persistence confirmed.

SEVERITY:
High

CONFIDENCE:
Medium

ESCALATION:
Required

Part 62 — Example — Suspicious Service Change

DISPOSITION:
Suspicious Host Activity

OBSERVATION:
A service was restarted shortly after an unexpected privileged
session.

CHANGE RECORD:
None identified.

PROCESS TELEMETRY:
Incomplete.

SEVERITY:
Medium

CONFIDENCE:
Medium

NEXT STEP:
Review configuration evidence and endpoint telemetry before
concluding unauthorized modification.

Part 63 — Assign Severity

Consider:

• privileged/root access

• host criticality

• successful unauthorized authentication

• service modification

• account changes

• multiple hosts

• outbound communication

• sensitive data access

• persistence evidence

• security-control outcome

Part 64 — Assign Confidence Separately

Example:

Severity:
High

Confidence:
Medium

means potential impact is significant, but malicious intent or full scope remains unconfirmed.

Part 65 — Determine Escalation

Escalate when:

• successful login from clearly unexpected source

• privileged activity is unexplained

• root-context actions are unauthorized

• multiple accounts targeted

• service-account misuse suspected

• suspicious process activity is correlated

• multiple hosts affected

• containment may be required

Part 66 — Create the Escalation Summary

ESCALATION SUMMARY

Case ID:

Host:

User:

Account Type:

Priority:

Source:

First Observed:

Last Observed:

Successful Authentication:
Yes / No

Privileged Activity:
Yes / No

Related Processes:

Related Services:

Network Activity:

Current Scope:

Confirmed Evidence:

Unconfirmed Concerns:

Recommended Next Steps:

Evidence References:

Part 67 — Consider Containment Recommendations

Depending on case evidence:

• revoke active SSH session

• reset account credential

• rotate affected service credential

• temporarily disable account

• restrict source system

• isolate endpoint/server

• preserve relevant logs

• increase monitoring

Do not perform these actions automatically.

Part 68 — Preserve Evidence Before Containment

Preserve:

• authentication logs

• relevant journal entries

• sudo events

• service events

• endpoint telemetry

• network logs

• case timeline

Use:

Preserve → Contain → Continue Investigation

where authorized and practical.

Part 69 — Build the Linux Evidence Register

Evidence ID	Source	Description
EV-LNX-01	SIEM	Original alert
EV-LNX-02	auth.log	Failed SSH events
EV-LNX-03	auth.log	Successful SSH event
EV-LNX-04	sudo	Privileged activity
EV-LNX-05	journal	Service event
EV-LNX-06	Firewall	Network correlation

Part 70 — Build the Final Linux Investigation Register

CASE ID:

HOST:

CRITICALITY:

ALERT:

PRIMARY USER:

ACCOUNT TYPE:

SOURCE:

SOURCE EXPECTED:
Yes / No / Unknown

FAILED AUTHENTICATION:

SUCCESSFUL AUTHENTICATION:

AUTHENTICATION METHOD:

SESSION ACTIVITY:

SUDO ACTIVITY:

USER SWITCHING:

ACCOUNT CHANGES:

GROUP CHANGES:

PROCESSES:

SERVICES:

SCHEDULED ACTIVITY:

SERVICE ACCOUNTS:

NETWORK CORRELATION:

DNS CORRELATION:

CHANGE CONTEXT:

RELATED HOSTS:

RELATED USERS:

TELEMETRY GAPS:

POSITIVE CONTROLS:

DISPOSITION:

SEVERITY:

CONFIDENCE:

PRIORITY:

ESCALATION:

CONTAINMENT:

REMAINING QUESTIONS:

Part 71 — Mission Challenge

Complete:

LAB INFORMATION

Lab:
Linux Authentication & System Log Investigation

Case ID:

Analyst:

Date:


INITIAL ALERT

Alert:

Host:

User:

Source:

Time:

Severity:

Priority:


HOST CONTEXT

Hostname:

IP:

Distribution:

Role:

Criticality:

Owner:

Expected Administrators:

Expected Services:


LOG SOURCES

auth.log / secure:

journalctl:

syslog/messages:

Endpoint:

Network:


SSH AUTHENTICATION

Failed Attempts:

First Failure:

Last Failure:

Successful Authentication:

Authentication Method:

Source:

Expected Source:

Session Opened:

Session Closed:


SOURCE ANALYSIS

Source IP:

Hostname:

Zone:

Known Asset:

Owner:

Other Accounts Targeted:

Expected Administration Source:


PRIVILEGE ACTIVITY

sudo Observed:
Yes / No

User:

Target User:

Command:

Expected:

su / User Switching:

Privileged Group Changes:


ACCOUNT ACTIVITY

Accounts Created:

Accounts Modified:

Passwords Changed:

Unexpected Changes:


PROCESS ACTIVITY

Process:

Parent:

User:

Command:

Expected:

Network Activity:


SERVICE ACTIVITY

Service:

Action:

Actor:

Expected:

Change Ticket:

Operational Impact:


SERVICE ACCOUNTS

Account:

Expected Source:

Observed Source:

Interactive Login:

Failures:

Assessment:


NETWORK CORRELATION

SSH Source:

Outbound Connections:

DNS Activity:

Firewall Evidence:

Unexpected Destinations:


CHANGE MANAGEMENT

Maintenance Window:

Ticket:

Administrator:

Events Match:
Yes / No


SCOPE

Affected Hosts:

Affected Users:

Privileged Accounts:

Service Accounts:

Affected Services:


TELEMETRY GAPS

Gap 01:

Impact:

Gap 02:

Impact:


POSITIVE CONTROLS

SSH Restriction:

MFA:

sudo Logging:

Service Account Restrictions:

Network Monitoring:

Other:


FINAL ASSESSMENT

Disposition:

Severity:

Confidence:

Priority:

Known Impact:

Escalation:
Yes / No

Containment Recommended:
Yes / No

Remaining Questions:

Ready for Phishing Email Investigation:
Yes / No

Part 72 — What Not to Do

Do not:

Assume failed SSH means brute force

Assume successful SSH means compromise

Assume sudo means privilege escalation vulnerability

Assume root-context command means malicious activity

Assume bash, curl, wget, python, or ssh are malicious tools

Assume service restart means persistence

Assume new account means backdoor

Ignore service accounts

Ignore automation

Ignore maintenance windows

Ignore source-network context

Ignore missing logs

Modify original log evidence

Delete suspicious logs

Clear shell history

Reset credentials without authorization

Disable user accounts without authorization

Stop services without approval

Modify sudoers

Change SSH configuration

Run exploit code

Perform password attacks

Generate suspicious activity unnecessarily

Declare Linux compromise from a single log entry

The professional rule is:

Correlate identity, privilege, service, and network activity before assigning malicious intent.

Troubleshooting

/var/log/auth.log Does Not Exist

Your distribution may use:

/var/log/secure

or primarily:

journalctl

Identify the correct source.

SSH Failures Exist but No Successful Authentication

Do not assume compromise.

Record the failures and assess source/user scope.

sudo Event Appears but Command Is Missing

Document the telemetry limitation.

Do not invent the command.

Source IP Belongs to a Jump Host

Identify the originating user/session where telemetry allows it.

Do not attribute the activity directly to the jump host itself.

Service Restart Appears Suspicious

Check:

• deployment

• monitoring

• scheduled restart

• maintenance window

• application failure

before escalating.

Account Failures Occur at Exact Intervals

Investigate service accounts, scheduled tasks, and stale credentials.

User Says Activity Was Authorized

Correlate against:

• source

• timestamps

• privilege activity

• change records

before closing.

Evidence Requirements

Capture:

Evidence 01

Original alert.

Evidence 02

Case scope.

Evidence 03

Linux Host Profile.

Evidence 04

Log Source Inventory.

Evidence 05

Failed SSH events.

Evidence 06

Successful SSH event.

Evidence 07

Authentication Sequence.

Evidence 08

Source Authentication Profile.

Evidence 09

Other-user correlation.

Evidence 10

Other-source correlation.

Evidence 11

Session activity.

Evidence 12

Sudo Activity Register.

Evidence 13

User-switching evidence.

Evidence 14

Account Activity Register.

Evidence 15

Group-change evidence.

Evidence 16

Process evidence.

Evidence 17

Service Activity Register.

Evidence 18

System-journal evidence.

Evidence 19

Scheduled activity where applicable.

Evidence 20

Service Account Baseline.

Evidence 21

Network Correlation Register.

Evidence 22

DNS correlation.

Evidence 23

Change-management correlation.

Evidence 24

Host Correlation Matrix.

Evidence 25

Master Linux Timeline.

Evidence 26

Hypothesis Matrix.

Evidence 27

Host/User/Privilege Scope.

Evidence 28

Telemetry Gap Register.

Evidence 29

Linux Security Control Matrix.

Evidence 30

Risk assessment.

Evidence 31

Escalation Summary.

Evidence 32

Linux Evidence Register.

Evidence 33

Linux Investigation Register.

Evidence 34

Mission Challenge worksheet.

Mission Deliverables

Complete:

• Linux case opened

• investigation window defined

• host context documented

• available Linux logs identified

• failed SSH events analyzed

• successful SSH event analyzed

• authentication method identified

• source host investigated

• other accounts searched

• other source systems searched

• session activity reviewed

• sudo activity reviewed

• user switching assessed

• account changes reviewed

• group changes reviewed

• process evidence analyzed where available

• service activity reviewed

• system journal reviewed

• scheduled activity reviewed where applicable

• service accounts assessed

• network evidence correlated

• DNS evidence correlated

• change-management context reviewed

• master timeline completed

• multiple hypotheses evaluated

• host scope determined

• user scope determined

• privilege scope determined

• service scope determined

• telemetry gaps documented

• positive controls documented

• disposition assigned

• severity assigned

• confidence assigned separately

• escalation decision documented

• containment recommendation documented

• final Linux Host Investigation Report completed

Lab Report Template

# Lab 05 — Linux Authentication & System Log Investigation

## Executive Summary

## Mission Objective

## Case Information

## Initial Alert

## Investigation Scope

## Linux Host Context

## Log Sources

## SSH Authentication Analysis

### Failed Authentication
### Successful Authentication
### Authentication Method
### Source Analysis

## Session Activity

## Sudo & Privilege Activity

## User Switching

## Account Management

## Group Membership Changes

## Process Activity

## Service Activity

## System Journal Analysis

## Scheduled Activity

## Service Account Analysis

## Network Correlation

## DNS Correlation

## Change Management Correlation

## Master Linux Timeline

## Hypothesis Analysis

## Host Scope

## User Scope

## Privilege Scope

## Service Scope

## Telemetry Gaps

## Positive Security Controls

## Final Disposition

## Severity

## Confidence

## Escalation Decision

## Containment Recommendations

## Evidence Register

## Remaining Questions

## Limitations

## Conclusion

Knowledge Check

Question 1 — Does repeated SSH failure prove brute-force activity?

No.

It may represent user error, stale credentials, automation, or a service-account issue.

Question 2 — Does a successful SSH login prove compromise?

No.

The source, user, timing, authentication method, and subsequent activity must be evaluated.

Question 3 — Does sudo usage mean a privilege-escalation vulnerability was exploited?

No.

Authorized administrators commonly use sudo.

Question 4 — Why are service accounts important during Linux investigations?

Because they often authenticate automatically and may generate unusual-looking behavior that is either legitimate automation or a meaningful security anomaly.

Question 5 — Why correlate SSH activity with network telemetry?

It helps confirm the source and determine whether unusual communications occurred after authentication.

Question 6 — Why review change-management records?

They can explain privileged login, package deployment, service restart, and other activity that may otherwise appear suspicious.

Question 7 — What should you do when command-line evidence is missing?

Document the telemetry gap and avoid guessing what was executed.

Question 8 — Does a service restart prove persistence?

No.

It may represent normal maintenance, recovery, or deployment activity.

Question 9 — Why separate severity and confidence?

The potential impact can be high even when the evidence is incomplete.

Question 10 — What is the central question of this lab?

“What occurred on the Linux system, which identity and services were involved, and does the log evidence support normal administration, an operational issue, or suspicious host activity?”

Skills Achieved

After completing this lab, you should understand:

• Linux security-log investigation

• SSH authentication analysis

• failed-login analysis

• successful-login analysis

• source attribution

• session analysis

• sudo investigation

• privilege-context analysis

• user switching

• Linux account-management investigation

• group-change analysis

• process-context analysis

• service investigation

• system-journal analysis

• scheduled-activity investigation

• service-account analysis

• network correlation

• DNS correlation

• change-management correlation

• Linux incident timeline reconstruction

• hypothesis-driven investigation

• telemetry-gap documentation

• host/user/privilege scoping

• SOC escalation

• Linux incident reporting

Professional Takeaway

A weak Linux investigation looks like:

Failed SSH
    ↓
Successful SSH
    ↓
sudo
    ↓
Server Compromised

A professional investigation looks like:

Alert
  ↓
Identify Host
  ↓
Identify User
  ↓
Identify Source
  ↓
Analyze Authentication
  ↓
Analyze Session
  ↓
Analyze sudo / Privilege
  ↓
Analyze Process + Service
  ↓
Correlate Network Activity
  ↓
Check Change Context
  ↓
Build Timeline
  ↓
Evaluate Hypotheses
  ↓
Determine Scope
  ↓
Evidence-Based Conclusion

Always distinguish:

Failed SSH
   ≠
Attack

Successful SSH
   ≠
Compromise

sudo
   ≠
Privilege Escalation Vulnerability

root Command
   ≠
Malicious Command

Service Restart
   ≠
Persistence

New User
   ≠
Backdoor Account

curl / wget / python
   ≠
Malicious Tool Automatically

Unknown Source
   ≠
Malicious Source Automatically

One Suspicious Linux Event
   ≠
Host Compromise

The strongest Linux investigator does not ask only:

“Which command was executed?”

They ask:

“Who authenticated, from where, using what method, what privilege was available, what happened during the session, what changed on the system, and does the entire sequence match expected administrative behavior?”

What’s Next?

➡️ Lab 06 — Phishing Email Investigation

In the next lab, you will move from host authentication analysis into one of the most common SOC investigations: phishing.

You will investigate:

• sender and recipient

• message headers

• message routing

• reply-to discrepancies

• SPF

• DKIM

• DMARC

• URLs

• attachment metadata

• sender/domain context

• user interaction

• IOC extraction

• endpoint correlation

• DNS/web correlation

• other recipients

• incident scope

• phishing classification

• containment and escalation recommendations

The methodology becomes:

Email → Header → Sender → URL/Attachment → IOC → User Exposure → Correlation → Scope → Disposition

The central question will be:

“Is this email legitimate, spam, suspicious, or malicious—and did any user interaction turn the message into a security incident?”