Trying to follow GNU Parallel as job queue with named pipes with GNU parallel 20201222, I run into issues of parallel not executing the last commands piped into it via tail -n+0 -f
.
To demonstrate, I have 3 terminals open:
# terminal 1
true > jobqueue
tail -n+0 -f jobqueue | parallel
# terminal 2
tail -n+0 -f jobqueue | cat
Adding a single small test command to the queue:
# terminal 3
echo "echo test" >> jobqueue
Only terminal 2 prints "echo test", gnu parallel does not output anything.
# terminal 3
for i in `seq 10`; do echo "echo $i" >> jobqueue; done
Only terminal 2 prints "echo 1", ..., "echo 10" (one in each line), gnu parallel does not output anything.
# terminal 3
for i in `seq 100`; do echo "echo $i" >> jobqueue; done
Terminal 2 prints "echo 1", ..., "echo 100". Terminal 1 prints the lines "test", "1", ..., "10", "1", ..., "99", the last line "100" is missing.
Rerunning tail -n+0 -f jobqueue | parallel
outputs all up to "99". Rerunning this with --resume --joblog log
appended, outputs one more line ("100") but then also lags behind once new lines are added to joblog
. For GNU parallel 20161222, the initial run only gets to line "84".
How can I force gnu parallel to flush its input queue on every line?
与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…