Semi-Automated Memory Saving
I have been on a quest this weekend to find a way to automate, or at least semi-automate, the process of saving memorified posts for offline viewing and backup purposes.
This was inspired by my own process of cleaning out my LiveJournal in the aftermath of the move to the Russian servers and TOS. I had almost 1900 memories in many categories. I was using the memories feature, among several other things, as a de facto personal fanfic archive, sorted into categories (especially fic from the days before tags on LJ, and before A03). Personally, I backed up my own by opening each memory and printing posts individually to PDF. It took several days of full-time work. It worked, but I would not recommend that method to anyone hoping to avoid RSI and/or nervous breakdown. A lot of my friends are in the same boat, and I figured there had to be a better solution.
I tried and discarded several options, including writing a client dedicated to the purpose. This proved unworkable because LJ's API does not interact with memories and does not allow export of comments for posts you don't administer (ie, friends' posts you've memorified, which in many cases people want specifically due to conversations or information found in the comments).
After a lot of different avenues of enquiry, I have now reached a workable solution using a few different Firefox extensions. It doesn't remove all the fiddling, but it cuts it to a manageable level, and makes me wish I'd done it before backing up my own. I suggest, if you're going to do this, you do it quickly, because a lot of your memories are probably disappearing as people delete their accounts.
Limitations:
1. Part of this method is to open all memories in a memory category, in new tabs. If you have many, many memories in a single category, this could crash your browser and/or computer. Test the method on a smaller category first, and if you're happy with how it works for you, invest some time separating them manually into smaller categories (ie, Funny Memes might become Funny Memes Part 1 and Part 2).
2. This method saves pages as MHT files, which are basically all-in-one webpage files readable by most browsers. However, embedded streaming content such as videos typically are not able to be saved.
3. I tested this on PC. It should work on Firefox for Mac and Linux as well, but there may be minor differences in how things behave.
Before you begin: You will need Firefox, and two Firefox add-ons, Greasemonkey and UnMHT. Install the add-ons and restart Firefox. Edit April 2018: The latest version of Firefox, Quantum, killed off a lot of add-ons, including unMHT. You can search for an alternative MHT saver plug-in, or downgrade Firefox temporarily (or permanently, as many have) to pre-Quantum.
Step 1: Make a One-Click Expand-All-LJ-Comments button on the bookmark toolbar
This may not work for free users, because it relies on Expand links on the page that are visible to paid users. I don't know if they're still buried in the code and usable by the script if you're a free user. If it doesn't work for you, that's fine, just go on to Step 2.
1. In Firefox, create a new bookmark by hitting Ctrl-D.
2. In the dialogue box, change the location to the Bookmark Toolbar. Leave everything else the same for the moment (it will bookmark the page loaded when you hit Ctrl-D, possibly this one). Click OK.
3. The Bookmark Toolbar will load beneath the address bar, with the bookmark you just made. Right-click it and click Properties.
4. Change the name to "One-click LJ comments" (or similar, doesn't need to be exact).
5. Change the location to:
javascript:(function(){var%20elinks%20=%20document.querySelectorAll('.b-tree-twig-1%20.b-leaf-actions-expandchilds%20>%20a');%20for%20(var%20i=0;%20i
6. Click Save.
7. To test, go to an LJ page with nested comments, such as this one. Then, once the page is loaded, click the bookmark you just made. All Expand links on the page will, not surprisingly, expand.
Credit for this method belongs here.
Step 2: Add a Greasemonkey script to open all memories in a category as tabs
It is your responsibility to understand what the script below does, and assure yourself it doesn't do anything nasty. The comments in the code explain what each part of it does. Credit for the underlying script belongs here.
1. Copy the code below:
// ==UserScript==
// @name LJMemoriesLinkOpen
// @require http://ajax.googleapis.com/ajax/libs/jquery/2.1.3/jquery.min.js
// @namespace http://www.yourDomainName.com/GreaseMonkey/
// @description Opens all links to *.html pages in new tabs when a page loads. In practice, since this script only operates on LiveJournal memory pages, it will open all entries linked from a LiveJournal memory category.
// @include *livejournal.com/tools/memories*
// @grant none
// ==/UserScript==
this.$ = this.jQuery = jQuery.noConflict(true); //make a safe instance of jQuery
var baseUrl = ".html";
//The $ function is an alias to the jQuery function. It's shorter, and most commonly used.
//You pass a selector string into the $ function to select some elements on the page.
//We want to find all "a" tags (anchors) whose href attribute contains the baseUrl. *= is the "atribute contains" selector.
//See: http://api.jquery.com/category/selectors/ for other selectors.
//To make it clearer, after adding in the baseUrl, this will be $( "a[href*='*.html']" ). LJ only uses HTML suffixes on entries.
var matchingLinks = $( 'a[href*="' + baseUrl + '"]' );
//Use jQuery to loop over each link and run a function for each one.
//$(this) returns a jQuery wrapper for the current node.
//This is nice because we can use jQuery functions on it, like attr, which returns the value of the specified attribute.
//If the page contains more than one link to the same href, it will be opened multiple times.
$(matchingLinks).each(
function(index)
{
window.open( $(this).attr( "href" ) );
}
);
2. Click the Greasemonkey icon on your Firefox toolbar, then New User Script.
3. Click "Use script from clipboard."
4. Click "Save."
Beware! The script is now active on all LiveJournal memory pages, and will automatically open every entry linked from that page in a new tab. This could crash your computer if you were to open a very large memory category before you're ready to work with it, as discussed above under Limitations. If you aren't yet ready to proceed, go to Greasemonkey - Manage Scripts and disable the LJMemoriesLinkOpen script until you're ready to start backing them up.
Step 3: Load and prepare one of your LJ categories
1. Close all your other tabs, and open your LJ memories page. (The script is active with this page, but nothing will happen yet because the main memories page doesn't link to any *.html pages (entries). It only links to category pages, which are suffixed BML).
2. Your memory categories will be listed. Click a category, preferably a small one to start with. The number of entries in each category is listed.
3. The category page will open, and then all the entries will gradually open as new tabs. Make sure you let them fully load.
4. Click into each tab and do a quick check for three things:
4a. Is the entry working? (That is, the entry has not been deleted or purged, or locked down so it is no longer visible to you). If the entry isn't working, close the tab.
4b. Has the page fully loaded? Or does it have missing images? Decide whether you care enough to reload. It's possible that it just didn't load because you had so many tabs going at once, in which case, reloading will solve it, but if the image is irrelevant, you simply might not care. Of course, the image may also be a dead link.
4c. Ask yourself if you care about the comments for this particular post. You will probably know straight away whether or not that's the case. (Fanfic? Probably no, unless you wrote it. Commentversations? Hell yes). If not, skip the next bit and move on to the next tab. But if you do care about the comments for a given entry, look further. Does the entry have multiple pages of comments? If so, right-click the additional comment page(s) and open them in new tabs. Also, does the entry have nested comments that need to be expanded? If so, use the One-Click Expand button to show them, or if that doesn't work for you, expand them manually.
You now have an entire category open in tabs, with comments expanded where desired and dead links removed, ready to be saved.
Step 4: Save your open tabs with UnMHT
1. Once you've gone through each of the tabs, click the UnMHT button in your toolbar (probably close to Greasemonkey), and click Save Multiple Tabs As MHT.
2. A dialogue box appears. By default, all open tabs will be selected.
3. Under Folder, name a new folder that corresponds to the memory category (eg HotSlashFic *cough*). Keep the format as Separated and state as Current State.
4. Click Save.
5. You will see the progress onscreen. It will take a little while if you have many tabs open.
6. When you're done, you'll have a folder full of MHT files that correspond to the entries you opened.
7. If you like, you can test the faithfulness of what's been saved by right-clicking an MHT file, and use "Open with" to try it in a browser you don't normally use (and therefore where you aren't logged in, don't have cached pages, etc).
Step 5: Auto-close the tabs
1. Right-click on your Memory Category tab (probably the one at the far left) and click Close Tabs To The Right.
2. A confirm dialogue will appear. Click Close Tabs.
By the way, you might be able to use this method to prevent your browser crashing if you open a large memory category in error and accidentally open a hundred tabs. You'll have to act fast.
Step 6. Repeat Steps 3-4-5 for remaining categories.
Step 7. (Mandatory) Disable the LJMemoriesLinkOpen script.
You must disable the script that opens all the memory entries at this point, otherwise you will be in for a very nasty surprise the next time you wander into a friend or community's memories.
1. Click Greasemonkey - Manage User Scripts.
2. In the Add-Ons Manager, find LJMemoriesLinkOpen, and click Disable. If you have no foreseeable further use for it, you might like to remove it completely.
Step 8. (Optional) Delete your memories.
If you're cleaning out your journal and wish to delete your memories, you can make the process easier with the following steps. Make sure you disable the LJMemoriesLinkOpen script (Step 7) before doing this.
1. Go to your memories page.
2. Click on a category.
3. In the address bar, add on to the end of the URL, &multidelete=1
4. Click enter. The category page will reload, this time with tickboxes to select multiple memories for deletion.
5. Select as many (or all) memories desired, and then Delete Selected.
6. Repeat for the other categories.
Good luck, all, and many thanks to cschick and ruakh for helping me out along the way. This post will be made public, so you can link to it freely.
This was inspired by my own process of cleaning out my LiveJournal in the aftermath of the move to the Russian servers and TOS. I had almost 1900 memories in many categories. I was using the memories feature, among several other things, as a de facto personal fanfic archive, sorted into categories (especially fic from the days before tags on LJ, and before A03). Personally, I backed up my own by opening each memory and printing posts individually to PDF. It took several days of full-time work. It worked, but I would not recommend that method to anyone hoping to avoid RSI and/or nervous breakdown. A lot of my friends are in the same boat, and I figured there had to be a better solution.
I tried and discarded several options, including writing a client dedicated to the purpose. This proved unworkable because LJ's API does not interact with memories and does not allow export of comments for posts you don't administer (ie, friends' posts you've memorified, which in many cases people want specifically due to conversations or information found in the comments).
After a lot of different avenues of enquiry, I have now reached a workable solution using a few different Firefox extensions. It doesn't remove all the fiddling, but it cuts it to a manageable level, and makes me wish I'd done it before backing up my own. I suggest, if you're going to do this, you do it quickly, because a lot of your memories are probably disappearing as people delete their accounts.
Limitations:
1. Part of this method is to open all memories in a memory category, in new tabs. If you have many, many memories in a single category, this could crash your browser and/or computer. Test the method on a smaller category first, and if you're happy with how it works for you, invest some time separating them manually into smaller categories (ie, Funny Memes might become Funny Memes Part 1 and Part 2).
2. This method saves pages as MHT files, which are basically all-in-one webpage files readable by most browsers. However, embedded streaming content such as videos typically are not able to be saved.
3. I tested this on PC. It should work on Firefox for Mac and Linux as well, but there may be minor differences in how things behave.
Before you begin: You will need Firefox, and two Firefox add-ons, Greasemonkey and UnMHT. Install the add-ons and restart Firefox. Edit April 2018: The latest version of Firefox, Quantum, killed off a lot of add-ons, including unMHT. You can search for an alternative MHT saver plug-in, or downgrade Firefox temporarily (or permanently, as many have) to pre-Quantum.
Step 1: Make a One-Click Expand-All-LJ-Comments button on the bookmark toolbar
This may not work for free users, because it relies on Expand links on the page that are visible to paid users. I don't know if they're still buried in the code and usable by the script if you're a free user. If it doesn't work for you, that's fine, just go on to Step 2.
1. In Firefox, create a new bookmark by hitting Ctrl-D.
2. In the dialogue box, change the location to the Bookmark Toolbar. Leave everything else the same for the moment (it will bookmark the page loaded when you hit Ctrl-D, possibly this one). Click OK.
3. The Bookmark Toolbar will load beneath the address bar, with the bookmark you just made. Right-click it and click Properties.
4. Change the name to "One-click LJ comments" (or similar, doesn't need to be exact).
5. Change the location to:
javascript:(function(){var%20elinks%20=%20document.querySelectorAll('.b-tree-twig-1%20.b-leaf-actions-expandchilds%20>%20a');%20for%20(var%20i=0;%20i
6. Click Save.
7. To test, go to an LJ page with nested comments, such as this one. Then, once the page is loaded, click the bookmark you just made. All Expand links on the page will, not surprisingly, expand.
Credit for this method belongs here.
Step 2: Add a Greasemonkey script to open all memories in a category as tabs
It is your responsibility to understand what the script below does, and assure yourself it doesn't do anything nasty. The comments in the code explain what each part of it does. Credit for the underlying script belongs here.
1. Copy the code below:
// ==UserScript==
// @name LJMemoriesLinkOpen
// @require http://ajax.googleapis.com/ajax/libs/jquery/2.1.3/jquery.min.js
// @namespace http://www.yourDomainName.com/GreaseMonkey/
// @description Opens all links to *.html pages in new tabs when a page loads. In practice, since this script only operates on LiveJournal memory pages, it will open all entries linked from a LiveJournal memory category.
// @include *livejournal.com/tools/memories*
// @grant none
// ==/UserScript==
this.$ = this.jQuery = jQuery.noConflict(true); //make a safe instance of jQuery
var baseUrl = ".html";
//The $ function is an alias to the jQuery function. It's shorter, and most commonly used.
//You pass a selector string into the $ function to select some elements on the page.
//We want to find all "a" tags (anchors) whose href attribute contains the baseUrl. *= is the "atribute contains" selector.
//See: http://api.jquery.com/category/selectors/ for other selectors.
//To make it clearer, after adding in the baseUrl, this will be $( "a[href*='*.html']" ). LJ only uses HTML suffixes on entries.
var matchingLinks = $( 'a[href*="' + baseUrl + '"]' );
//Use jQuery to loop over each link and run a function for each one.
//$(this) returns a jQuery wrapper for the current node.
//This is nice because we can use jQuery functions on it, like attr, which returns the value of the specified attribute.
//If the page contains more than one link to the same href, it will be opened multiple times.
$(matchingLinks).each(
function(index)
{
window.open( $(this).attr( "href" ) );
}
);
2. Click the Greasemonkey icon on your Firefox toolbar, then New User Script.
3. Click "Use script from clipboard."
4. Click "Save."
Beware! The script is now active on all LiveJournal memory pages, and will automatically open every entry linked from that page in a new tab. This could crash your computer if you were to open a very large memory category before you're ready to work with it, as discussed above under Limitations. If you aren't yet ready to proceed, go to Greasemonkey - Manage Scripts and disable the LJMemoriesLinkOpen script until you're ready to start backing them up.
Step 3: Load and prepare one of your LJ categories
1. Close all your other tabs, and open your LJ memories page. (The script is active with this page, but nothing will happen yet because the main memories page doesn't link to any *.html pages (entries). It only links to category pages, which are suffixed BML).
2. Your memory categories will be listed. Click a category, preferably a small one to start with. The number of entries in each category is listed.
3. The category page will open, and then all the entries will gradually open as new tabs. Make sure you let them fully load.
4. Click into each tab and do a quick check for three things:
4a. Is the entry working? (That is, the entry has not been deleted or purged, or locked down so it is no longer visible to you). If the entry isn't working, close the tab.
4b. Has the page fully loaded? Or does it have missing images? Decide whether you care enough to reload. It's possible that it just didn't load because you had so many tabs going at once, in which case, reloading will solve it, but if the image is irrelevant, you simply might not care. Of course, the image may also be a dead link.
4c. Ask yourself if you care about the comments for this particular post. You will probably know straight away whether or not that's the case. (Fanfic? Probably no, unless you wrote it. Commentversations? Hell yes). If not, skip the next bit and move on to the next tab. But if you do care about the comments for a given entry, look further. Does the entry have multiple pages of comments? If so, right-click the additional comment page(s) and open them in new tabs. Also, does the entry have nested comments that need to be expanded? If so, use the One-Click Expand button to show them, or if that doesn't work for you, expand them manually.
You now have an entire category open in tabs, with comments expanded where desired and dead links removed, ready to be saved.
Step 4: Save your open tabs with UnMHT
1. Once you've gone through each of the tabs, click the UnMHT button in your toolbar (probably close to Greasemonkey), and click Save Multiple Tabs As MHT.
2. A dialogue box appears. By default, all open tabs will be selected.
3. Under Folder, name a new folder that corresponds to the memory category (eg HotSlashFic *cough*). Keep the format as Separated and state as Current State.
4. Click Save.
5. You will see the progress onscreen. It will take a little while if you have many tabs open.
6. When you're done, you'll have a folder full of MHT files that correspond to the entries you opened.
7. If you like, you can test the faithfulness of what's been saved by right-clicking an MHT file, and use "Open with" to try it in a browser you don't normally use (and therefore where you aren't logged in, don't have cached pages, etc).
Step 5: Auto-close the tabs
1. Right-click on your Memory Category tab (probably the one at the far left) and click Close Tabs To The Right.
2. A confirm dialogue will appear. Click Close Tabs.
By the way, you might be able to use this method to prevent your browser crashing if you open a large memory category in error and accidentally open a hundred tabs. You'll have to act fast.
Step 6. Repeat Steps 3-4-5 for remaining categories.
Step 7. (Mandatory) Disable the LJMemoriesLinkOpen script.
You must disable the script that opens all the memory entries at this point, otherwise you will be in for a very nasty surprise the next time you wander into a friend or community's memories.
1. Click Greasemonkey - Manage User Scripts.
2. In the Add-Ons Manager, find LJMemoriesLinkOpen, and click Disable. If you have no foreseeable further use for it, you might like to remove it completely.
Step 8. (Optional) Delete your memories.
If you're cleaning out your journal and wish to delete your memories, you can make the process easier with the following steps. Make sure you disable the LJMemoriesLinkOpen script (Step 7) before doing this.
1. Go to your memories page.
2. Click on a category.
3. In the address bar, add on to the end of the URL, &multidelete=1
4. Click enter. The category page will reload, this time with tickboxes to select multiple memories for deletion.
5. Select as many (or all) memories desired, and then Delete Selected.
6. Repeat for the other categories.
Good luck, all, and many thanks to cschick and ruakh for helping me out along the way. This post will be made public, so you can link to it freely.