Snippets

DavidC Does parallel exit before its child processes are done?

Created by David Cuddihy last modified
confirm.py
Raw 
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
import pandas as pd

sss="""0 -rw-r--r--  1 dcuddihy domainusers    0 2015-11-18 13:29:28.157439000 -0500 1
0 -rw-r--r--  1 dcuddihy domainusers    0 2015-11-18 13:29:33.653439000 -0500 2
0 -rw-r--r--  1 dcuddihy domainusers    0 2015-11-18 13:29:38.729439000 -0500 3
0 -rw-r--r--  1 dcuddihy domainusers    0 2015-11-18 13:29:44.025439000 -0500 4
0 -rw-r--r--  1 dcuddihy domainusers    0 2015-11-18 13:29:49.281439000 -0500 5
0 -rw-r--r--  1 dcuddihy domainusers    0 2015-11-18 13:29:54.917439000 -0500 6
0 -rw-r--r--  1 dcuddihy domainusers    0 2015-11-18 13:29:59.929439000 -0500 7
0 -rw-r--r--  1 dcuddihy domainusers    0 2015-11-18 13:30:05.821439000 -0500 8
0 -rw-r--r--  1 dcuddihy domainusers    0 2015-11-18 13:30:11.549439000 -0500 9
0 -rw-r--r--  1 dcuddihy domainusers    0 2015-11-18 13:30:16.653439000 -0500 10
0 -rw-r--r--  1 dcuddihy domainusers    0 2015-11-18 13:30:22.649439000 -0500 done"""

ticks = [(x.split()[9], pd.Timestamp(x.split()[7])) for x in sss.splitlines()]
deltas = [(ticks[i][0], ticks[i+1][0], ticks[i+1][1]-ticks[i][1]) for i in range(1,len(ticks)-1)]
['%s was create %s after %s' % (d[1],d[2],d[0]) for d in deltas]

## Returns this:
['3 was create 0 days 00:00:05.076000 after 2',
 '4 was create 0 days 00:00:05.296000 after 3',
 '5 was create 0 days 00:00:05.256000 after 4',
 '6 was create 0 days 00:00:05.636000 after 5',
 '7 was create 0 days 00:00:05.012000 after 6',
 '8 was create 0 days 00:00:05.892000 after 7',
 '9 was create 0 days 00:00:05.728000 after 8',
 '10 was create 0 days 00:00:05.104000 after 9',
 'done was create 0 days 00:00:05.996000 after 10']
 
 ## Note that 'done' was created a full 5.99 seconds after '10'
 ## ...which indicates that parallel didn't return until after
 ## the "sleep 5" completed execution.
results
Raw 
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
/tmp$ ls -lastr --time-style=full-iso
total 20
4 drwxr-xr-x 23 root     root        4096 2015-11-09 16:31:25.131100000 -0500 ..
4 drwxrwxrwt  2 root     root        4096 2015-11-18 10:39:01.033439000 -0500 .X11-unix
4 drwxrwxrwt  2 root     root        4096 2015-11-18 10:39:01.037439000 -0500 .ICE-unix
4 -rwxr-xr-x  1 dcuddihy domainusers   75 2015-11-18 13:29:18.189439000 -0500 test.sh
0 -rw-r--r--  1 dcuddihy domainusers    0 2015-11-18 13:29:28.157439000 -0500 1
0 -rw-r--r--  1 dcuddihy domainusers    0 2015-11-18 13:29:33.653439000 -0500 2
0 -rw-r--r--  1 dcuddihy domainusers    0 2015-11-18 13:29:38.729439000 -0500 3
0 -rw-r--r--  1 dcuddihy domainusers    0 2015-11-18 13:29:44.025439000 -0500 4
0 -rw-r--r--  1 dcuddihy domainusers    0 2015-11-18 13:29:49.281439000 -0500 5
0 -rw-r--r--  1 dcuddihy domainusers    0 2015-11-18 13:29:54.917439000 -0500 6
0 -rw-r--r--  1 dcuddihy domainusers    0 2015-11-18 13:29:59.929439000 -0500 7
0 -rw-r--r--  1 dcuddihy domainusers    0 2015-11-18 13:30:05.821439000 -0500 8
0 -rw-r--r--  1 dcuddihy domainusers    0 2015-11-18 13:30:11.549439000 -0500 9
0 -rw-r--r--  1 dcuddihy domainusers    0 2015-11-18 13:30:16.653439000 -0500 10
0 -rw-r--r--  1 dcuddihy domainusers    0 2015-11-18 13:30:22.649439000 -0500 done
4 drwxrwxrwt  4 root     root        4096 2015-11-18 13:31:12.697439000 -0500 .
test.sh
Raw 
1
2
3
#!/usr/bin/env bash
seq 1 10 | parallel "touch {};sleep 5"
touch /tmp/done

Comments (0)

HTTPS SSH

You can clone a snippet to your computer for local editing. Learn more.