Skip to content

Instantly share code, notes, and snippets.

@greenantdotcom

greenantdotcom/Taming application logs

Created January 23, 2019 22:05
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save greenantdotcom/8774c20ceb0b0494a114611f95558647 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save greenantdotcom/8774c20ceb0b0494a114611f95558647 to your computer and use it in GitHub Desktop.
Download ZIP
Splunk wrangling - SSNR and AKG
Raw
Taming nginx logs
(index=sp-tomcat-ssnreuse-prdidx source="/logs/ssnreuse-tomcat_app_aws-prd_us-*-2/app/nginx/logs/access.log") OR (index=sp-tomcat-ctds-prdidx source="/logs/ctds-tomcat_app_prodstg_us-*-2/app/nginx/logs/access.log" request="GET /server-status HTTP/1.1")
| eval log_source_type=if(match(index,"sp-tomcat-ctds-prdidx"), "status_log", "access_log")
| eval responseTime__raw=coalesce('upstream.responseTime', 'request_time')
| eval responseTime=case(responseTime__raw="-",0.00,isNull(responseTime__raw),0.00,1=1,tonumber(responseTime__raw))
| eval responseTimeMS=responseTime*1000
| eval event_type=case(
match(request,"server-status"),
"nginx.server-status",
match(request,"/health/local"),
"health_check.local",
match(request,"/health/full"),
"health_check.full",
match(request,"/v1/fetchReturnNeedsHoldStatus"),
"api.v1.fetchReturnNeedsHoldStatus",
match(request,"/v1/listAccountsAndNotifications"),
"api.v1.listAccountsAndNotifications",
match(request,"/v1/updateSsnLookupData"),
"api.v1.updateSsnLookupData",
match(request,"/v1/"),
"api.v1.*",
1=1,
"unknown"
)
| stats count, perc50(responseTimeMS), perc90(responseTimeMS), perc95(responseTimeMS), perc99(responseTimeMS) by event_type, datacenter
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment