SED can make some things SO much easier
If you have a linux machine and haven’t ever made use of sed (stream editor) you’re missing out on a great automation utility. I’ve saved myself probably 20 hours of manual editing with about an hour of work TWICE today. Here’s how…. over on the North Carolina Genealogy site I was opening forums for each county in the state of North Carolina (100 counties.) Now, I could have gone through and typed out a description, slug (address) and name for each one, but that looked too tedious. So…
I copied and pasted the list of counties to a text file (there was a carriage return after each) so I had one per line. Then I made a second file which I used tr to convert from upper case to lower case (since I wanted lower case “slugs”) I then went through and used sed to convert a space to an underscore. Now, I could have done this by hand because the only county with a space in the name is New Hanover, but…. it was easy as the following…
cat county_lowercase.txt | sed 's/ /_/' > county_lowercase_nospace.txt
So, the next step was crafting the slugs…
cat county_lowercase_nospace.txt | sed 's/$/-county-nc-genealogy-queries/' > slugs.txt
I checked the forum list table for my forum software through phpmyadmin and it was a pretty simple structure, about 8 fields…. a unique id which I set to NULL in my database, forum name, description, slug, forum order, number of posts, number of topics…. I set the last two to 0 throughout (100 lines), the forum order I started a bit past the current forum order numbers that were listed in the current database and I copied and pasted from the textfile the slugs.
So, next I crafted the forum titles….
cat county_list.txt | sed 's/$/ County, NC Genealogy Query Forum/' > forum_titles.txt
And of course, those get copied and pasted into the correct column. By the way, the $ above tells sed to put the text at the end of each line so that the county name comes first.
Then the descriptions, again I cut and paste and things look good. From the spreadsheet now I export to csv and import via phpmyadmin and check, lo and behold I now have 100 forums correctly added.
The other big thing though was a script I use to retrieve rss feeds. Each page was to have one uniquely named that would be included via a php include call in the page.
So… again sed to the rescue.
#!/bin/bash
for name in `cat county_lower_nospace.txt`;
do
cat template_gen.php | sed "s/template_search/$name+county%22+nc/" > "$name"_gen.php
done
VERY easy and it worked like a charm…. 100 php scripts created, one for each county in …. 5 seconds? Nice.
Finally to top things off, I used sed to make the links to insert in to the pages… again all 100. So, it turned into a simple copy paste for each one.
Thanks sed.. I don’t know how else I could have done this that didn’t involve hours of menial/tedious editing.
For more information on some of the neat textfile tricks you can do with sed… take a look at this sed guide.