NOTE: This is a question I found on StackOverflow which I’ve archived here, because the answer is so effing phenomenal.
If you are not into long explanations, see [Paolo Bergantino’s answer][2].
| #################################################################### | |
| # Create compute environment and job queue for AWS Batch with Boto3 | |
| # Daisuke Miyamoto | |
| # 20200330 | |
| # | |
| import boto3 | |
| import botocore.waiter | |
| from botocore.exceptions import WaiterError | |
| import time |
| import multiprocessing | |
| from multiprocessing.dummy import Pool as ThreadPool | |
| import numpy as np | |
| def my_multipro(items, func, max_cpus=12): | |
| """Do an embarrassingly parallel task using multiprocessing. | |
| Use this for CPU bound tasks. |
| # IF YOU INCUR HUGE COSTS WITH THIS OR IT BREAKS DON'T BLAME ME License | |
| # This is a throw-away script I wrote to pull the json events for all of the streams from a cloudwatch log | |
| # For some reason, the naive way to do vpc network logging does logging to different streams in a cloudwatch | |
| # log based on interface. | |
| # Great for diagnosing lots of things, and generating verbose logs, but for the broad-stroke analysis I was doing, | |
| # all I really wanted was the basic data. This would have been easier if I had logged to s3, but I did not see a | |
| # way to do that in 2 clicks. | |
| group_name = 'CHANGEME' |
NOTE: This is a question I found on StackOverflow which I’ve archived here, because the answer is so effing phenomenal.
If you are not into long explanations, see [Paolo Bergantino’s answer][2].
| -- show running queries (pre 9.2) | |
| SELECT procpid, age(clock_timestamp(), query_start), usename, current_query | |
| FROM pg_stat_activity | |
| WHERE current_query != '<IDLE>' AND current_query NOT ILIKE '%pg_stat_activity%' | |
| ORDER BY query_start desc; | |
| -- show running queries (9.2) | |
| SELECT pid, age(clock_timestamp(), query_start), usename, query | |
| FROM pg_stat_activity | |
| WHERE query != '<IDLE>' AND query NOT ILIKE '%pg_stat_activity%' |