Recovering Files with Runtime Software’s GetDataBack

I like to think I am fairly decent at data recovery, or I used to be, though I have not really had a need in more recent times, and being “fairly decent” is mostly, “Knowing where to find what tools to use.” It’s not like I am out here replacing drive power boards or, I don’t know, manually laser imaging disc platters or something. I used to use a piece of software called R-Studio. I am not sure it even really exists anymore. Whatever the case, the key I have for that is from a version from the mid-2000s.

It’s… Like 20 years old…

Fuck I am old.

….

Anyway, my experience with R-Studio was pretty great, I mostly used it in my old office IT job. We bought a copy when one night the automation system we used to run the TV station had a drive failure. Reprogramming it would have been a bitch (but doable). I pulled the PC out, we thought about our options and went with this data recovery route. The drive was able to be reimaged onto a good drive and the system was up and running again.

Yay.

Over the years I used it pretty regularly to recover crashed laptops from coworkers. Generally just the Documents and PST files in those cases though. It was also useful for my own drives and drives of people I knew.

The years, however, made me a bit jaded about being IT Support for everyone I had ever met. There are quite a few jokes about this around online, and it’s true. These days, I basically will just “play dumb” because if you fix one problem for one neighbor, now you will be fixing everyone’s PC issues.

And let me tell you, that is often self-inflicted on what those issues are, which is worse than dealing with real technical problems. Sorry I can’t recover your Pentium, no I can’t make it run Facebook Faster, it’s just too old.

Anyway, at one point I lost some family photos when an external drive crashed on me. This caused two things to happen. One, I will never ever buy a USB drive again, or at least not trust them with important data. I am talking about things like those Seagate drives that have a TB or more that plug into the wall and your PC. It doesn’t help that they also are designed in a way that the plastic housing can’t be removed easily and often requires DESTROYING the housing to recover them. Two, it caused me to get serious about backup, and the cloud. I have used a few different services over the years, but for a long time, I have been at the point where my entire house could burn to the ground and my data would be safe and recoverable.

I have a cloud-synced backup with some versioning and the whole “recycle bin” option in One Drive and I keep an incremental backup on a hard drive in a static bag in a fire safe in the house. Plus all the data is in a RAID on my NAS.

But I don’t back up EVERYTHING. That would be too much data because I am a bit of a data hoarder. And a lot of the non-essential stuff gets stored on an assortment of “dodgy drives” that I have collected over the years from a variety of places. For example, I am currently using my previous “Degraded per Synology” NAS drive as a base to build a PLEX server. I mean, it didn’t technically fail, Synology just didn’t like it, and so I replaced it in the NAS, and now it’s a 4TB drive that, is probably mostly still sort of good.

Anyway, I had one of those USB drives that I mentioned above as having sworn off that was previously working for this task. My wife goes to a LOT of estate sales as part of her business and I often tag along. At one I found this drive stuck buried on a bookshelf, so I bought it for like $5. I figured it was probably good, and it worked, for a bit. But as they usually seem to do, this sucker decided to die on me.

I tried to see if I could get it to read with some Linux tools, but I had little luck. I went online looking for data recovery tools and remembered using straight search is a bad idea for this because it’s going to be 99% “articles” from companies recommending their own software. So I went to Reddit, and had GetDataBack suggested.

I downloaded it and it managed to detect the drive and files. I decided to bite the bullet and paid for the full version, which is not cheap at $80, but I have some other drives I could run through this and it’s a “lifetime license” so I will eventually feel like I am getting my money’s worth.

And it’s working just great. The only real problem I am having is that it won’t recover to a network drive and my PC’s internal drive does not have 2TB of space on it, so I have to recover things in chunks, then copy it over to its final storage place. Well, that and the normal issues that come with a failed/failing drive where sometimes things get hung up and just don’t recover. The interface is straightforward and nice as well, though not super pretty.

Like I said, I am sure I will get my money’s worth. I have a drive that was my brother’s somewhere that I can try to recover. I also have some NVME drives that I will need to get a USB hook up for, but I wouldn’t mind trying to get data back off of those.

Dead Memory Cards and Using Docker

More often that it feels like it should, something in technology breaks or fails. I find that this can be frustrating, but often ultimately good, especially for learning something new, and forcing myself to clean up something I’ve been meaning to clean up. I have a Raspberry Pi I’ve been using for a while for several things as a little web server. It’s been running probably for years, but something gave out on it. I’m not entirely sure it’s the SD card or the Pi itself honestly, because I’ve been having a bit of trouble trying to recover through both. It’s sort of pushed me to try a different approach a bit.

But first I needed a new SD card. I have quite a few, most are “in use”. I say “in use” because many are less in use and more, underused. This has resulted in doing a bit of rebuild on some other projects to make better use of my Micro SD cards. The starting point was a 8 GB card with just a basic Raspbian set up on it.

So, for starters, I found that the one I have in my recently set up music station Raspberry Pi is a whopping 128gb. Contrary to what one might thing, I don’t need a 128gb card in my music station, the music is stored on the NAS over the network. It also has some old residual projects on it that should really be cleaned out.

So stuck the 8GB card in that device and did the minor set up needed for the music station. Specifically, configure VLC for Remote Control over the network, then add the network share. Once I plugged it back into my little mixer and verified I could remote play music, I moved on.

This ended up being an unrelated side project though, because I had been planning on getting a large, speedy, Micro SD card to stick in my Retroid Pocket. So I stuck that 128GB card in, the Retroid and formatted it. This freed up a smaller, 32GB card.

I also have a 64GB that is basically not being used in my PiGrrl Project I decided to recover back for use. The project was fun, but the Retroid does the same thing 1000x better. So now it’s mostly just a display piece on a shelf. Literally an overpriced paperweight. I don’t want to lose the PiGrrl configuration though, because it’s been programmed up to work with the small display and IO Control Inputs. So I imaged that card off.

In the end though, I didn’t end up needing those Micro SD cards though, I opted for an alternative option to replace the Pi, with Docker on my secondary PC. I’ve been meaning to better learn Docker, though I still find it to be a weird and obtuse bit of software. There are a handful of things I care about restoring that I used the Pi for.

  • Youtube DL – There seem to be quite a few nice Web Interfaces for this that will work much better than my old custom system.
  • WordPress Blog Archives – I have exported data files from this but I would like to have it as a WordPress Instance again
  • FreshRSS – My RSS Reader. I already miss my daily news feeds.

YoutubeDL was simple, they provided a nice basic command sequence to get things working.

The others were a bit trickier. Because the old set up died unexpectedly, The data isn’t easily exported for import, which means digging out and recovering off of the raw database files. This isn’t the first time this has happened, but its a lot bigger pain, which isn’t helped by not being entirely confident in how to manipulate Docker.

I still have not gotten the WordPress archive working actually. I was getting “Connection Reset” errors and now I am getting “Cannot establish Database connection” issues. It may be for nothing after the troubles I have had dealing with recovering FreshRSS.

I have gotten FreshRSS fixed though. Getting it running in Docker was easy peasy. Getting my data back, was… considerably less so. It’s been plaguing me now when I try to fix it for a few weeks now, but I have a solution. It’s not the BEST solution, but it’s… a solution. So, the core thing I needed were the feeds themselves. Lesson learned I suppose, but I’m going to find a way to automate a regular dump of the feeds once everything is reloaded. I don’t need or care about favorited articles or the articles contents. These were stored in a MySQL database. MySQL, specifically seems to be what was corrupted and crashed out on the old Pi/Instance because I get a failed message on boot and i can’t get it to reinstall or load anymore.

Well, more, I am pretty sure the root cause is the SD card died, but it affected the DB files.

My struggle now, is recovering data from these raw files. I’ve actually done this before on a surver crash years ago, but this round has lead to many many hurdles. One, 90% of the results looking up how to do it are littered with unhelpful replies about using a proper SQL dump instead. If I could open MySQL, I sure as hell would so that. Another issue seems to be that the SQL server running on the Pi was woefully out of date, so there have been file compatibility problems.

There is also the issue that the data may just flat out BE CORRUPTED.

So I’ve spun up and tried to manually move the data to probably a dozen instances of MySQL and MariaDB of various versions, on Pis, in Docker, on WSL, in a Linux install. Nothing, and I mean NOTHING has worked.

I did get the raw data pulled out though.

So I’ve been brute forcing a fix. Opening the .ibd file in a text editor gives a really ugly chuck of funny characters. But, strewn throughout this, is a bunch of URLs for feeds and websites and well, mostly that. i did an open “Replace” in Notepad++ that stripped out a lot of the characters. Then I opened up Pycharm, I did a find and replace with blanks on a ton of other ugly characters. Then I write up this wuick and dirty Python Script:

# Control F in Notepad++, replace, extended mode "\x00"
# Replace "   " with " "
# replace "https:" with " https:"
# rename to fresh.txt

## Debug and skip asking each time
file = "fresh.txt"
## Open and read the Log File supploed
with open(file, encoding="UTF-8") as logfile:
    log = logfile.read()

datasplit = log.split(" ")
links = []

for each in datasplit:
    if "http" in each:
        links.append(each)

with open("output.txt", mode="w", encoding="UTF-8") as writefile:
    for i in links:
        writefile.write(i+"\n")

Which splits everything up into an array, and skims through the array for anything with “http” in it, to pull out anything that is a URL. This has left me with a text file that is full of duplicates and has regular URLs next to Feed URLS, though not in EVERY case because that would be too damn easy. I could probably add a bunch of conditionals to the script to sort out anything with the word “feed” “rss”, “atom” or “xml” and get a lot of the cruft removed, but Fresh RSS does not seem to have a way to bulk import a text list, so I still get to manually cut and paste each URL in and resort everything into categories.

It’s tedious, but it’s mindless, and it will get done.

Afterwards I will need to reset up my WordPress Autoposter script for those little news digests I’ve been sharing that no one cares about.

Slight update, I added some filtering ans sorting to the code:

# Control F in Notepad++, replace, extended mode "\x00"
# Replace "   " with " "
# replace "https:" with " https:"
# rename to fresh.txt


## Debug and skip asking each time
file = "fresh.txt"
## Open and read the Log File supploed
with open(file, encoding="UTF-8") as logfile:
    log = logfile.read()

datasplit = log.split(" ")
links = []

for each in datasplit:
    if "http" in each:
        if "feed" in each or "rss" in each or "default" in each or "atom" in each or "xml" in each:
            if each not in links:
                links.append(each[each.find("http"):])

links.sort()

with open("output.txt", mode="w", encoding="UTF-8") as writefile:
    for i in links:
        writefile.write(i+"\n")

Hard Drive Woes Part 2

This post is a follow up to my previous Dead Hard Drive post.

I used to hassle with PC hardware a LOT more than I currently do. I’ve kind of worked my way out of that gig honestly. I am at a point where I can afford shit for starters, mostly, so I’m not trying to cobble together workable machines from random parts. I also got tired of doing tech support for people, so I basically just, sort of hide that I can, because when people find out you can “fix computers”, now you’re vacuuming out 50 years of dust from a Pentium 1 in your backyard for a neighbor who refuses to just buy literally any cheapest machine at Wal-Mart for an infinite performance boost.

“Back in my day!” (fist shaking), you could pretty much just slap any drive with an Operating system in any machine and it would boot. Sometimes it would boot into an ugly driverless environment because it was ripped from another machine, but that was fixable. Things seem more complicated these days. I’m not blaming UEFI, and all that more secure BIOS stuff, but it’s a likely culprit. I think that better security is good, it just, is also part of why I can’t more conveniently fix my damn PC.

I say Conveniently, because that’s the core issue. I can still EASILY do it. It’s just… not convenient.

Shortly after messing with Linux a bit for troubleshooting, I did a bit of set up to use it as the main driver but, decided to just go back to Windows. I downloaded a fresh recovery image, sliced the Linux partition down to 500GB and reinstalled Windows.

I like Linux. I use Linux, almost daily, if not daily. It’s great for automation tasks and running server software and all that. It, kind of really sucks as a desktop OS. Don’t get me wrong, it’s usable, especially for simpler needs (literally anything not Gaming or Video/Photo Editing). I have run Linux as the sole OS on many machines, mostly laptops, and lots of Pis and Servers. I’ve used Linux off and on for over 20 years now. The problem here is, the main use case for my “Kick ass gaming rig” is well, gaming. Half the games I had slated “to play” from Steam are not available in Linux. I set up Hero Launcher for GOG and Epic, but like, my cloud saves didn’t work, and Fortnite doesn’t work and the whole thing felt a little off. Graphics also felt a little off, even though I did switch to using the official proprietary NVidia drivers.

Anyway, I went back to Windows. I spent an eternity downloading drivers and doftware and getting things set up properly. Unfortunately, the secondary drive I was now using as my primary, is just too slow to handle the needs of a lot of games as well. I had to roll Fortnite back to DirectX 11 for example, because it would take like 10 minutes to drop into a match because it would load shaders or some shit. For anyone not aware of how Fortnite works, it’s online, in an arena of players. If you drop in 10 minutes late, your character will have already landed in the map and probably be dead or dying.

So I bit the bullet and bought a new NVME drive. I planned to eventually, I just, did it sooner.

I went and downloaded Clonezilla to just mirror the Hard Drive to the NVME drive, which worked, but things would not boot.

There are plenty of possible solutions online, with recovery mode. I tried a few of them. But in the end, I have opted to just, reinstall Windows, again.

Which means redownloading drivers and shit… again….

I might be able to pull the Steam Downloads over before wiping the secondary drive, but I am not sure Epic will let me do that. Unfortunately, the larger games are from Epic, with Fortnite, Death Stranding and Final Fantasy 7R in that list.

It’s all, very easy.

It’s just all, very inconvenient.

Also, just because, and maybe for future reference, the install needs:

  • Network Driver – For some reason it doesn’t work on the generic.
  • TUF Gaming Amoury Crate – The motherboard seems to load this, and it find and installs all the drivers, which is nice, despite the cheesy name.
  • Windows Update
  • Color Scheme to Dark, no transparency
  • Firefox – Browser of choice, then log into sync and let it pull all my stuff in.
  • Steam
  • Epic
  • Visual Studio Code
  • Change One Drive settings to not sync everything but only some things.
  • Log into the Microsoft Account so One Drive and Office work, since no network driver means local account log in only at first
  • Share X – For Screenshots to folders
  • Display Fusion – For rotating desktop wallpaper
  • Synergy KVM – So I can connect to my other PC\
  • EVGA Flow Control – For the cooler
  • Remove all the cruft from the start menu, remove the apps list and recent files
  • Add a dozen network drives to File Exporer
  • Discord
  • Firestorm Viewer

A Dead Hard Drive

I came down last night to drop some stuff off in the basement and shut the curtains, and sat down to check on something at my desktop PC, I don’t even remember what, and was slightly surprised to see that it was sitting at the BIOS Screen and not the Windows lock screen. My first assumption was, it did an update or something, and the cat was sitting on the keyboard, and cause it to enter the BIOS. They don’t usually sit on the keyboard, but it’s possible.

I rebooted the PC, and…. it just loaded the BIOS again.

Clearly something more than a cat issue.

Both the 1TB M2 NVMe drive and the 2TB add on drive were showing in the BIOS menu, but no boot options were showing available. In fact, it even specifically said something along the lines of “No boot options.” I tried resetting the BIOS settings back to factory default, I had toggled a few things so I could do virtualization, and it was no help.

I dug out a USB key with Linux and booted to that. I mostly wanted to see if I could access the drive still at all. This had to be done with an extremely weird and annoying bright yellow screen where everything was washed out. The Live OS would boot fine and look fine, until it actually got to the point of letting me do anything, when it suddenly seemed to give up it’s video driver causing everything to go wonky.

I managed to squint my way through it and the drive shows up, but it’s not accessible at all.

So I swapped out Ubuntu for a Windows Recovery USB Key. The recovery options (restore, recovery, etc) all failed. These gave a bit more information that the drive was “locked”. I tried a few more options at the command line that I found,

  • bootrec /fixMBR
  • bootrec /fixBoot
  • bootrec /rebuildBCD

But none of these changed anything. I could probably download and run a Windows ISO, but for the moment, I’ve decided on a different route. I booted back into Ubuntu, and just installed that on my 2TD spare drive. It would not take on the 1TB NVMe drive, and the 2TB secondary drive was just all games anyway, so nothing of value would be lost by wiping it clean.

I might, MIGHT just try running this way for a while, though it does have some disadvantages. Mostly, games. Almost everything I’ve been actively playing lately was through the Epic store. And a lot of the games I planned to get to in Steam, don’t work in Linux. There are ways to get them to work though, which I want to look into, but I have not had time yet. I do know Fortnite is flat out not going to work. Not a huge loss, I am kind of getting tired of it again anyway. It has some strict Anti-Cheat which won’t run in any sort of emulated environment.

I also still have my old desktop too I can use. So well, Fortnite may not be out completely, it just, won’t run quite as nice. In fact, I can probably run most of the stuff I want on that machine that won’t work directly in Ubuntu.

Another thing worth mentioning, I am not really out anything file wise. A handful of downloaded files for “TODO” projects that I could download again. I basically never work with files directly on any particular system anymore, it’s always files on the NAS or files in One Drive. The only thing I really lost were the handful of custom Stable Diffusion Embeddings I had created, and I have been meaning to try to rebuild better versions of those anyway.

It will be interesting to see how performance is compared to Windows though. This PC is a pre built gaming PC, so I am sure it’s been somewhat optimized for use with Windows. I have not had a chance to really test it out in a Linux environment at all yet, but I’m interested to see the results. I’d already been toying with the idea of running Linux on this machine but I was worried about how it would handle things like the water cooler. I already don’t have the ability to control the lights on my Keyboard and Mouse, but there may be software to do that available if I look into it.

All in all, I am irritated that the drive died, but I’ve taken it much more in stride than one might expect. I will probably poke at the Windows system some more as well though. The drive doesn’t really act like it’s dead, more like, it’s got some sort of software glitch going on.

ActivityPub and WordPress

The whole concept of the “Fediverse” and decentralization has been gaining a lot of traction lately, especially with the recent bull shit around Twitter. I’ve had a Mastodon account for a few years and have tried a few times to get into the whole ActivityPub decentralized thing a few times over the year after hearing it discussed on the TWIT Podcast a few times.

With the recent news of Automattic (owners of WordPress) buying the most popular Activity Pub plug in, I’ve decided to give it another go. So I’ve re-enabled the plug in, and actually made a better effort to look into what it means, and it, was, surprisingly pretty simple. I am sure it can do more, but basically, both of my blogs can now be followed on other ActivityPub based Fedi-verse systems. Specifically, the two accounts, for now, are:

@JoshMiller@lameazoid.com

and

@JoshMiller@bloggingintensifies.com

I followed both, basically, the easy way, is to slap one or the other in the search box on Mastodon or a Mastodon app, and they show up like any other account. I checked, and though the post is very long, and the images don’t show properly, sure enough, it shows up in Tusky.

i could see some fun possibilities here, though I don’t know that anyone cares enough to actually follow any of it in the first place. The first I’ll probably try, i have a system that posts blobs of links that I share via Fresh RSS daily (if there is anything to share). These currently just post with me as the author. However, I could easily change this to another user, say, “LinkBot” and give it a separate feed, so @linkbotLinkBot@bloggingintensifies.com could be followed, just for those links.

I could make other sub accounts like this too, maybe I could have one that’s just a feed of random food photos. I could set up something on Lameazoid for toy photos or video games screen shots.

I will add in that the concept is neat, but I also am not sure I really prefer it to just straight RSS. It seems a bit overly complicated when “Really Simple Syndication” is a thing.