The "Reality Uncovered" forum (founded by Ryan Dube and Stephen Broadbent) was one of the primary places for critical analysis of the Project SERPO saga and for delving into issues regarding UFO disinformation, the Aviary, and various UFO hoaxes. There were numerous relatively detailed discussions relating to Rick Doty, Ron Pandolfi and others.
With the kind blessing of one of the former owners of the "Reality Uncovered" forum, Ryan Dube, I've been attempting to create and share a searchable PDF archive of posts to that forum.
While the results are far from perfect (and involve considerable duplication for various technical reasons that do not seem worth resolving), I've uploaded 43,904 pages of searchable PDFs of posts to the "Reality Uncovered" forum (in about 3,000 separate files).
No one responded positively when I asked on Facebook and elsewhere if anyone had an archive of posts to that forum. That would have made life easier. :) In the apparent absence of anything better, I've downloaded an incomplete set of pages from the Wayback Machine's Internet Archive. Many pages of that forum (which used to be at realityuncovered.net) are preserved in the Wayback Machine archive, but sadly that archive is not easily searched. Thus, I thought it worth making pages of posts to that UFO forum available in one or two easily searched formats (including as PDF files, as I did with a Rendlesham UFO forum a few years ago).
After installing Ruby (a first for me...), I used the free "Wayback Machine Downloader" available on Github to download over 7,000 archived files from the realityuncovered.net website (most of those files being html files for individual pages of posts to that UFO forum).
I've had some help from a Facebook contact, who prefers not to be named, and spending quite a bit of time (okay, a _wholly_ unreasonable amount of time...) learning about the use of Regular Expressions to find and replace links. Even after that work, only about 50% of the internal links work, but at least searching the collection of PDFs is possible.
To make the archive as complete as possible, I have not sought to eliminate all duplication (since the only ways that I could devise to eliminate duplication resulted in a loss of some content.
Anyway, on a more positive note, this searchable PDF archive has resulted in my being able to find most of the things I wanted to find so I thought other UFO researchers might also benefit from it.
For anyone interested in the technical details of the steps that I took to get at least some of the internal links working, after downloading the material from the Wayback Machine using the software I mentioned above:
Replaced .php with _php
Added file extension .html
In Notepad++, under the Search menu select Find then “Regular Expression” and tick “matches newline” and then:
Replaced .*<a name="start_here"></a> with <a name="start_here"></a>
Replaced <script([^/script]*) with nothing
In Notepad++, under the Search menu select Fine then “Normal”, and then:
Replaced .php with _php
Replaced ?f= with %253ff%253d
Replaced start= with start%253d
Replaced t= with t%253d
Replaced sid= with sid%253d
Replaced & with %2526
Replaced ?avatar= with %253favatar%253d
In Notepad++, under the Search menu select Find then “Regular Expression” and tick “matches newline” and then:
Replaced _php([^"]*) with _php\1.html
Replaced .jpg.html with .jpg
Replaced .gif.html with .gif
Replaced viewtopic_php([^\.]*).html with viewtopic_php\1.pdf"
(I'm pretty sure there will be more efficient and effective ways of getting more of the internal links to work, but I don't know them...).
No comments:
Post a Comment