A Problem

I think I have a Bash problem.  What follows is an actual command from my history.

cat /usr/share/dict/words | fgrep -v "'" | perl -ne 'chomp($_); @b=split(//,$_); print join("", sort(@b))." ".$_."\n";' | tee lookup.txt | perl -pe 's/^([^ ]+) .*/\1/g' | awk '{ print length, $0 }' | sort -n | awk '{$1=""; print $0}' | uniq -c | sort -nr | egrep "^[^0-9]+2 " | awk '{ print length, $0 }' | sort -n | awk '{$1=""; print $0}' | perl -pe 's/[ 0-9]//g' | xargs -i grep {} lookup.txt | perl -pe 's/[^ ]+ //g' | tail -n2

It’s just so hard to bite the bullet, admit that the problem has grown in scope, and move it to its own Perl/Python script.  (P.S. The Guinness Book is wrong.  “Conservationalists” is not a real word.)

Edit: to those who are competing in the comments to improve (shorten) the above command: when pasting code, use the <code> tag to override Wordpress quote formatting.

Joey Comeau has a new book out based on Overqualified, which has long been one of my favorite things on the internet.  He writes cover letters to companies.  They each sound businesslike enough for the first paragraph or so, and then you gradually realize you are reading something that is in no way a normal cover letter.  An excerpt from one to Nintendo:

We need a new Mario game, where you rescue the princess in the first ten minutes, and for the rest of the game you try and push down that sick feeling in your stomach that she’s “damaged goods”, a concept detailed again and again in the profoundly sex negative instruction booklet, and when Luigi makes a crack about her and Bowser, you break his nose and immediately regret it. When Peach asks you, in the quiet of her mushroom castle bedroom “do you still love me?” you pretend to be asleep. You press the A button rhythmically, to control your breath, keep it even.

#2 (NeoPost), #28 (Phone surveys) and #58 (MySpace) are three of my favorites.

99 Responses to “A Problem”

  1. anonymous says:

    That is awesome.

  2. Aislin Kageno says:

    I like the idea of that applicant conducting phone surveys to see what new content should be featured on myspace, if his surveys are anything like the ones he conducts in #28. Obviously, if he opens with ‘What’s a clitoris?” he’ll close with “Put lesbians on myspace.”

  3. IQgryn says:

    You should really skip the cat and just give the filename straight to fgrep.

    Yes, that’s the first thing I thought upon reading you’re command. :P

  4. IQgryn says:

    Err…your. I apparently should not post while tired.

  5. ben says:

    My grep doesn’t like list entries starting with a “-”. I propose “xargs -i grep — {} lookup.txt”.

  6. Pox says:

    I see you learnt tee (and stripped out some egrep aliases) since I last saw that command. Excellent.

  7. MeanderingMind says:

    This book concept reminds me a lot of “Letters from a Nut” by Ted L. Nancy. My brother and I used to pour over it when we were younger. Nancy would publish both the bizarre letters he sent to various companies and institutions (with strange hotel room requests or pitching a new product) and the replies he received. This turned into a bit of a series with a couple of books now out following the same format. Joey Comeau’s letters seem more biting, dark, and clever. Nancy’s could sometimes grind as too simple or formulaic. From the examples you’ve given, I don’t expect that will be a problem with “Overqualified”. Thanks for the heads-up…

  8. Anonymous says:

    usually by about the third time i invoke perl in a single command i get the point and just turn the whole thing into a perl one-liner…

  9. Jouni says:

    Perhaps your problem is that you are trying to fit your task into just one pipeline instead of two:

    lookup.txt; \
    join lookup.txt lookup.txt | \
    awk '$2<$3 && $1=length($1)' | \
    sort -n | \
    sed 's/[0-9]* //' | \
    tail -n2

  10. Jouni says:

    Another problem is that you should define what transformations your blog engine does to comment texts… let’s try escaping the redirection signs:

    < /usr/share/dict/words \
    perl -F -lane "next if /'/;"'print join("", sort @F)." $_";' | \
    sort > lookup.txt; \
    join lookup.txt lookup.txt | \
    awk '$2<$3 && $1=length($1)' | \
    sort -n | \
    sed 's/[0-9]* //' | \
    tail -n2

  11. Smylers says:

    I couldn’t resist golfing the command. Here’s what I came up with:

    fgrep -v \' /usr/share/dict/words | perl -nle 'print length, "\t$_\t", sort split ""'' | sort -k 1nr -k 3 | uniq -f 2 -D | head -n 2 | cut -f 2

    Using something like a Guttman–Rosler transform it’s possible to avoid the external look-up file.

  12. Sweth says:

    If you’re golfing, you’re leaving way too much whitespace and using the long versions of a lot of idioms:

    perl -nle'!/47/&&print length,"\t$_\t",sort split""' /usr/share/dict/word|sort -k1nr -k3|uniq -f2 -d|head -n2|cut -f2

  13. Nate says:

    I really like those letters. #28 reminds me a lot of twocansandstring.com

  14. Nicholas Knight says:

    Aside from the inexplicable use of that painful language awk, that looks a lot like command lines I come up with every day.

  15. David says:

    Any reason to use ‘[^ ]‘ rather than ‘\S’ ?

    And my workmate is laughing at me because that was my only comment on reading the line

  16. SnakeDoc says:

    Conversationalists is a real word.

    Love,
    Your friendly neighborhood English teacher.

  17. SnakeDoc says:

    Doh! Conversationalists is a real word, but Conservationalists is not. I am a dyslexic English teacher, it turns out.

  18. anonymouser says:

    No worries, SnakeDoc. These things happen.

  19. Stefan says:

    I just can’t resist the urge to share that “Bash problem” in Turkish means “quite a problem”

  20. xkcd says:

    Smylers: nice.

    Sweth: You forgot to cut out words with apostrophes.

    I’m sorry the blog is replacing quotes and such in your commands — try wrapping them with (I'll edit a couple comments to do that)

  21. Obsidian says:

    Conservationalists isn’t a word…but conservationists is. :P

  22. Chfan says:

    Wow, that looks great. Apparently John Campbell of Pictures for Sad Children is reading it too, so if all my favorite webcomic artists are ten I guess I should…

  23. FupDuckTV says:

    That quote to Nintendo is AWESOME… I’ve always seen Peach as “damaged goods”. I guess that is why I never completed Super Mario Brothers.

  24. Not knowing awk or perl, I usually end up piping back and forth between bash and sed, using almost only sed’s s command and, with the bash part, echo and for, with the occasional =.
    I think you’ve just expanded my toolbelt.

    CAPTCHA: more pompous

  25. nosetip says:

    echo -e ‘impressiveness\npermissiveness’

  26. Phaedrus2401 says:

    I caught that QC reference in the comic today. Har-har-har.

    $ sudo rm -rf /bin

    (DO NOT WANT)

    Or on Windows:

    C:\> del C:\Windows\System32\HAL.DLL

  27. Phaedrus2401 says:

    Also, my school network has no internal security, so my favorite command is this:

    C:\> pskill \\NETWORK_ADDRESS explorer.exe

    Completely destroys their GUI and they have no idea what’s going on.

  28. Kim says:

    Turing test: flouting Senator

  29. noah says:

    That book also sounds a LOT like Please Let Me Help by Zack Sternwalker which is actually the funniest zine ever.

  30. tuckels says:

    I wish i had a lesbian.

  31. Munin says:

    And is that an SP reference I spot as well?

  32. Brandon says:

    I simply had to post because it seems recaptcha is promoting some new, homoerotic S&M sitcom called 2 1/2 manacles

  33. Daniel says:

    reCAPTCHA: dvorak NATURAL

  34. Doug Orleans says:

    You need a better word list:

    microphotographies
    photomicrographies

    Not a very satisfying pair, though.

  35. RaptorsAnaomous says:

    Dude that comic instules everone on the austistc spectrum….
    Just that you may not have a specrum disorder that dosent mean outher linux user with highspeed wongt

    (Raports 4 evar)
    (ps just use a c program to query aspell with you bash issue)

  36. Rover says:

    Hanners is my personal favorite :D
    The Mario excerpt reminds me of the Ramayana, the Hindu myth. Rama fights an entire war to save his wife from an alleged demon-king and after winning the war, he dumps her, believing her to be damaged goods. This is the guy some of my people worship…

  37. tahrey says:

    is there some proto-sentient AI running amok in here or something?
    have you somehow connected Bucket to the blag script?

  38. Matt Hickford says:

    Before you execute untrusted obfuscated code from comment threads, you should switch to an unprivileged user, ie. one of

    sudo su nobody
    sudo -u nobody -s

  39. Matt Hickford says:

    I think the quotes are messed up

  40. Aaron A. says:

    Joey Comeau is a talented and disturbed young man. His letters are great; there’s always the unspoken, but heavily implied, notion that the writer* grew up in an environment of constant passive-agressive resentment, or shared an intimate moment with his uncle in the toolshed when he was seven.

    re: xkcd #575 – Do any music applications actually support tagging in the manner suggested by today’s comic? I’d love to be able to label songs as SFW or not, or describe the song’s mood with more than one cookie-cutter phrase, or tag a band with its influences. OS doesn’t really matter; I have Vista, Ubuntu, and Mandriva at my disposal, and maybe Leopard (if I broke the Mac, my wife would kick me out until I bought her a new computer, plus some new games, accessories, and a box of really good chocolates.)

    * I don’t mean Joey Comeau, the author of /Overqualified/, but “Joey Comeau,” the character who applies for each of these jobs.

    captcha: Sunday Cubicles. So yeah, Peter, if you just could go ahead and come in…

  41. at upton says:

    #65 sums up everything I like most about Joey Comeau’s writing.

    (My CAPTCHA words are “keep feminist”!)

  42. simbr says:

    Just thought I’d pitch in and add that I’m seeing a re-captcha of “marching grande”, which instantly reminded me of strip #389

  43. nosetip says:

    where do you get YOUR wordlist?

    xkcd’s version:
    real 0m21.943s
    user 0m13.716s
    sys 0m2.693s

    Smylers’ version:
    real 0m2.499s
    user 0m2.373s
    sys 0m0.013s

    Now I know what a GRT is; thanks Smyler! And nice UNIX-foo too.

  44. Drakar2007 says:

    @Rover: speaking of the Ramayana, have you seen Sita Sings the Blues yet?

  45. Tinctorius says:

    tinctorius@grass:~$ ## Insert weird xkcd command here.
    grep: invalid option -- t
    Usage: grep [OPTION]... PATTERN [FILE]...
    Try `grep --help' for more information.
    grep: invalid option -- t
    Usage: grep [OPTION]... PATTERN [FILE]...
    Try `grep --help' for more information.
    grep: unknown directories method
    grep: OWeeimpr: invalid context length argument

    grep: conflicting matchers specified
    grep: OWegiiknrtu: invalid context length argument

    grep: unknown directories method
    grep: invalid option -- W
    Usage: grep [OPTION]... PATTERN [FILE]...
    Try `grep --help' for more information.
    grep: unknown directories method
    grep: unknown directories method
    grep: unknown directories method
    grep: unknown directories method
    grep: Naaaddeeiilllnnnrsst: invalid context length argument

    grep: unknown directories method
    grep: unrecognized option `--eeeeegiiiklnnnnrrsstvw'
    Usage: grep [OPTION]... PATTERN [FILE]...
    Try `grep --help' for more information.
    belastingverplichtingen
    betalingsverplichtingen

    I do not know what just happened. Too many dashes in my word list (Dutch), perhaps?


  46. import solution

    Hooray for python!

  47. plash says:

    Matt wins!

    CAPTCHA: Minutes studying

  48. spirov92 says:

    Nice! BTW what is this script supposed to do? Spewing?
    Randal: you should save such long things in a shell script, you know. I wouldn’t risk losing this if something happens to ~/.bash_history .
    Also, why not write the whole thing in Perl or python? I would probably write it in PHP, since it’s the only language I know well enough :)
    reCAPTCHA: delicacy pervert :D nice

  49. Sven Neumann says:

    Hi Randall, didn’t know how to reach u so I’ll drop a comment here. you GOTTA check out the Scarpar. I left a link in the “Website” field. Hope you agree ;)
    Sorry, I couldn’t help but think of you when I saw that vid.

Leave a Reply