Parameter expansion to loop over pairs: ${a%b} and ${a#b}

[karmag]

Hey, a command line community? That's a pretty neat thing to have.

So to introduce myself, here's something I figured out pretty recently. It's in one of those eyes-glazing-over parts of the shell man page (but I had a specific problem to solve so I had to figure it out). Behold! It is a piece of "parameter expansion" magic:

$ TEST='foo:bar' ; echo ${TEST%:*} ${TEST#*:}
foo bar

Ooh, exciting! No wait, it's not. But here's the problem I had: How can I loop over pairs of values in a script or one-liner?

The solution I came up with looked something like this:

$ for f in url1:filename1 url2:filename2 url3:filename3;
do URL=${f%:*}; FILENAME=${f#*:} ;
...
; done

...and so for each pair, URL and FILENAME gets set to their respective portion. (And you'll be glad to learn that I got my Youtube-mediated crime drama fix in the end.)

Comments

[karmag]

That is, for those of you here who has bash installed as the default shell. I shouldn't make the sloppy assumption that everyone reading this has...

[pne]

In any case, when using bash-isms, use a shebang of #!/bin/bash - some people use #!/bin/sh under the assumption that everyone uses (certain distributions of) Linux /bin/sh is bash.

If you want bash, say so explicitly, then you won't be bitten on systems where /bin/sh is the original sh, or something else like the Almquist shell (ash/dash).

[karmag]

I (mostly) agree. But on the other hand you could argue that doing so will make perfectly cromulent scripts fail for users that don't have bash at all. (Or don't have bash in /bin, like me.) That's still better than failing silently and doing the wrong thing, though.

A related question is how you can know that you're using a bashism in the first place. It seems that the man page doesn't point out what are extensions and what is POSIX – so if your /bin/sh is bash, how will you know? And to make things more exciting, when started with the --posix flag bash still accepts (at least some of) its own extensions, so you can't use that for testing. (But I don't have a very recent version, so perhaps this is no longer a problem.)

I guess the only half decent way out of this is to install a bunch of different shells (ash, ksh, zsh, ...) and test anything you don't already know is portable. Yay, more work for everyone.

For reference and/or comparison, here is the /bin/sh man page for my system. I don't think there are many that can beat NetBSD when it comes to being conservative and boring, so perhaps it can be useful in setting a lowest common denominator. :-)