xargs and apostrophes/spaces

[sophie]

If you've ever tried using xargs with a list of filenames, you've probably at some point come across errors like these:

xargs: unmatched single quote; by default quotes are special to xargs unless you use the -0 option

ls: cannot access /home/sophie/directory: No such file or directory
ls: cannot access with: No such file or directory
ls: cannot access spaces: No such file or directory
ls: cannot access in/blah.txt: No such file or directory

Both of these errors are due to xargs by default interpreting some characters as special; in the first one, it won't allow apostrophes, and in the second it treats spaces as if they meant separate arguments. And the -0 switch doesn't help at all if you're using newlines.

Both of these errors can be resolved very easily: just add the -d"\n" switch before the command you want xargs to execute. This tells xargs that you don't want xargs to mess with your input at all except to treat newline characters as delimiters. This time, you should find that both apostrophes and spaces are accepted properly.

Comments

[pauamma]

Or (if you're using find), instead of

find ... -print | xargs ...

use

find ... -print0 | xargs -0 ...

[pne]

+1

"find | xargs" (with GNU find and GNU xargs) should always be "find ... -print0 | xargs -0 ..." - I can't think of a case where this would be the wrong thing to do.

[sophie]

It definitely wouldn't be the wrong thing to do, no. But I don't know of a filesystem that allows newlines in filenames, so it wouldn't be wrong to do the delimiter flag either - and then you don't need to do the -print0 command at all, so it actually saves a bit of typing.

[pauamma]

-d is (IIRC) not POSIX, and not all xargs versions implement it. See eg, http://www.freebsd.org/cgi/man.cgi?query=xargs&apropos=0&sektion=0&manpath=FreeBSD+8.1-RELEASE&format=html.

[karmag]

Yeah, it doesn't work here.

(Anyone here know a lot about POSIX and portability? If no one else does, perhaps I should read up on it and try to post something.)

[pauamma]

I have the SUSv3 docs as dead electrons, but I never read them.

[jld]

But I don't know of a filesystem that allows newlines in filenames

All the usual Unix filesystems allow anything that isn't / or NUL, as far as I know.

[pne]

That was my state of knowledge, too - that those are the only two forbidden characters.

Sophie: try this -

$ touch 'file name
with a newline in it'

and you should have, er, a file name with a newline in it. (Try ls and ls -l - and try ls fil*, though that will probably fail due to argument quoting!)

[jadelennox]

Timely advice. I just 5 min. ago gave up on debugging a piece of crap written by a predecessor of mine:

/usr/bin/find "$@" -name '*.xml' -exec xml sel -N "namespacewenthere" -t -m '/xpathwenthere' -v '@elementhere' -o ' ' -o '{}' '{}' ';' | sed -E 's/([^[:space:]]*) (.*)/xml tr scripts\/badlynamedtransform.xsl -s filesize="`cat "\2" | wc -c`" "\2" > includes\/\1.foxml.xml/' | sh


That had a lot of unnecessary garbage and also crapped out on the hundreds of filenames with spaces. I tried switching it to print0 with xargs -0, but had timing/precedence problems with the backtick. To make matters worse, I'm writing the code on a Debian system that's getting run on somebody else's Snow Leopard system, and our versions of find and xargs aren't compatible. Just 5 min. ago I said "fuck it, I'm doing this in perl".

*sigh*

[sophie]

Did you end up doing it in Perl?

[pauamma]

http://www.catb.org/jargon/html/J/job-security.html (But obviously, it can only go so far.)

[vlion]

Perl. For when you just Want To Get The Job Done.

[razputnik]

Nice, I normally end up writing hideously long bash one liners with xargs because I am a bad person - so this should come in handy :)