'date' enhancement: flush in format string (original) (raw)

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

From: Darko Veberic
Subject: 'date' enhancement: flush in format string
Date: Thu, 11 Aug 2011 23:54:08 +0200
User-agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.18) Gecko/20110617 Thunderbird/3.1.11

dear developers,

with awk i have been processing large text files where the first field was a human-readable date that had to be converted to unix second. for this purpose i was using an awk pipe call

{d="date -u -d "$1" +%s"; d | getline t; close(d); print t,...}

where $1 was holding the date string. unfortunately the files contained on the order of million such lines and awk was able to process only ~2000 lines per second. the slowness was mostly due to the spawning of the "date" process for each line.

then it occurred to me that i could use "date" as a co-process in awk directly, spawning "date" only once and using the date's -f redirected to standard input:

BEGIN{d="date -u -f- +%s"} {print $1 |& d; d |& getline t; print t,...} END{close(d)}

unfortunately "date" is not flushing its output after each input line is processed, resulting in awk's getline deadlock.

enhancement request:

is it possible to add another % symbol to the date output format, say %f (not taken yet) which would trigger output flush, ie i would modify my example above into

date -u -f- +%s%f

this would really help in my case!

best regards, %^>

darko.

asst.prof.dr. Darko Veberic -------------- Laboratory for Astroparticle Physics University of Nova Gorica Vipavska 13, SI-5000 Nova Gorica, Slovenia mobile: +386 41 860 861 .phone: +386 5 3315 255 ...fax: +386 5 3315 385

Department for Theoretical Physics Jozef Stefan Institute, Jamova 39, SI-1000 Ljubljana, Slovenia

CERN, NA61/SHINE, 892 R-A16 Z02200 CH-1211 Geneve 23, Switzerland phone: +41 22 76 764 89