JSON utilities
Probably everyone who's spent more than five minutes working with web services and JSON has versions of these utilities. Here are mine.
The first one is very simple: it takes a bit of JSON (passed as an argument, or over stdin, in the usual Unix-filter way) and spits it out in a nicely indented format to aid comprehension. I call it jd, but I'm thinking of changing the name to dj.
#!/usr/bin/perl -w
use JSON;
use Data::Dumper;
$/ = undef;
while (<>) {
print Dumper(from_json($_));
}
Yes, that could have been a one-liner. Given input {"this":{"is":{"a":"test"}}}, it will produce
$VAR1 = {
'this' => {
'is' => {
'a' => 'test'
}
}
}; As should be apparent, the output is not JSON, but rather Perl's object literal syntax. Perhaps JSON would be better: I'll make that change if and when I feel the need for it. For now, you can read the output in with Perl, eval it, and then call to_json on the result.
Data::Dumper comes with the core Perl distribution, but you'll need to install the JSON module: on most Unixy systems, that's as simple as typing sudo cpan JSON at a command prompt, and accepting the default answer for most of the configuration questions if you've never used cpan before. You may also wish to install JSON::XS for MAX SPEEDS: this will require you to have a C compiler installed somewhere, but don't worry, the actual compilation bit is all handled automatically. Just type sudo cpan JSON::XS.
The second utility, which I call jf, is inspired by Mark Dominus' f program. It extracts fields from JSON objects: if it encounters a list, it maps over the list and keeps going.
#!/usr/bin/perl -w
use JSON;
$/ = undef; # slurp whole files at a time
if (scalar(@ARGV) < 1) { usage() }
my @fields;
while ((my $field = shift)) {
if ($field eq '--') { last; }
push @fields, $field;
}
while (<>) {
print to_json(descend(from_json($_), @fields));
}
sub descend {
(my $json, my @fields) = @_;
if (ref($json) =~ /ARRAY/) {
return [ map { descend($_, @fields) } @$json ];
} elsif (scalar(@fields) > 0) {
my $field = shift @fields;
return descend($json->{$field}, @fields);
} else {
return $json;
}
}
sub usage {
print "Usage: $0 field1 field2 .. -- file1 file2 ..\n";
print "or cat file1 file2 .. | $0 field1 field2 ..\n";
exit 2;
}
Let's use it to get the screen names of the users who posted the 20 most recent Twitter statuses.
pozorvlak@delight:~/bin$ curl -s http://twitter.com/statuses/public_timeline.json | jf user screen_name
["topamazonsaving","blastp","erikbolhuis","BlaiseThermador", "hoewtaioasg","nazaret","mrsbenedict","ericelftmann","perebarr", "wheowheo","manish_kayal","pjhanse","cheewei","knowsnotmuch", "khanoomche","AmyyVee","currentutc","thecatcancook","tulachaendler", "berlotti"]
Awesome. Note that this is legal JSON: if you want it pretty-printed, pipe the output to jd.
This is actually the second version of jf: the first version only took one field at a time. To get the effect above, we'd have had to do curl ... | jf user | jf screen_name. The idea of separating fields from input files with -- was borrowed from Aaron Crane.
I gratefully acknowledge the generosity of my employers (for whom I do not speak...) for allowing me to release these programs despite them being written during work hours. Thanks!
The first one is very simple: it takes a bit of JSON (passed as an argument, or over stdin, in the usual Unix-filter way) and spits it out in a nicely indented format to aid comprehension. I call it jd, but I'm thinking of changing the name to dj.
#!/usr/bin/perl -w
use JSON;
use Data::Dumper;
$/ = undef;
while (<>) {
print Dumper(from_json($_));
}
Yes, that could have been a one-liner. Given input {"this":{"is":{"a":"test"}}}, it will produce
$VAR1 = {
'this' => {
'is' => {
'a' => 'test'
}
}
}; As should be apparent, the output is not JSON, but rather Perl's object literal syntax. Perhaps JSON would be better: I'll make that change if and when I feel the need for it. For now, you can read the output in with Perl, eval it, and then call to_json on the result.
Data::Dumper comes with the core Perl distribution, but you'll need to install the JSON module: on most Unixy systems, that's as simple as typing sudo cpan JSON at a command prompt, and accepting the default answer for most of the configuration questions if you've never used cpan before. You may also wish to install JSON::XS for MAX SPEEDS: this will require you to have a C compiler installed somewhere, but don't worry, the actual compilation bit is all handled automatically. Just type sudo cpan JSON::XS.
The second utility, which I call jf, is inspired by Mark Dominus' f program. It extracts fields from JSON objects: if it encounters a list, it maps over the list and keeps going.
#!/usr/bin/perl -w
use JSON;
$/ = undef; # slurp whole files at a time
if (scalar(@ARGV) < 1) { usage() }
my @fields;
while ((my $field = shift)) {
if ($field eq '--') { last; }
push @fields, $field;
}
while (<>) {
print to_json(descend(from_json($_), @fields));
}
sub descend {
(my $json, my @fields) = @_;
if (ref($json) =~ /ARRAY/) {
return [ map { descend($_, @fields) } @$json ];
} elsif (scalar(@fields) > 0) {
my $field = shift @fields;
return descend($json->{$field}, @fields);
} else {
return $json;
}
}
sub usage {
print "Usage: $0 field1 field2 .. -- file1 file2 ..\n";
print "or cat file1 file2 .. | $0 field1 field2 ..\n";
exit 2;
}
Let's use it to get the screen names of the users who posted the 20 most recent Twitter statuses.
pozorvlak@delight:~/bin$ curl -s http://twitter.com/statuses/public_timeline.json | jf user screen_name
["topamazonsaving","blastp","erikbolhuis","BlaiseThermador", "hoewtaioasg","nazaret","mrsbenedict","ericelftmann","perebarr", "wheowheo","manish_kayal","pjhanse","cheewei","knowsnotmuch", "khanoomche","AmyyVee","currentutc","thecatcancook","tulachaendler", "berlotti"]
Awesome. Note that this is legal JSON: if you want it pretty-printed, pipe the output to jd.
This is actually the second version of jf: the first version only took one field at a time. To get the effect above, we'd have had to do curl ... | jf user | jf screen_name. The idea of separating fields from input files with -- was borrowed from Aaron Crane.
I gratefully acknowledge the generosity of my employers (for whom I do not speak...) for allowing me to release these programs despite them being written during work hours. Thanks!