[Blogging Intensifies]

Technology, Coding, Music, Life...

  • About
  • Friends
  • Micro Posts
  • Music
  • Photo Gallery
  • Post Series
  • What I Use

Technology

Dead Memory Cards and Using Docker

May 18, 2023

More often that it feels like it should, something in technology breaks or fails. I find that this can be frustrating, but often ultimately good, especially for learning something new, and forcing myself to clean up something I’ve been meaning to clean up. I have a Raspberry Pi I’ve been using for a while for several things as a little web server. It’s been running probably for years, but something gave out on it. I’m not entirely sure it’s the SD card or the Pi itself honestly, because I’ve been having a bit of trouble trying to recover through both. It’s sort of pushed me to try a different approach a bit.

But first I needed a new SD card. I have quite a few, most are “in use”. I say “in use” because many are less in use and more, underused. This has resulted in doing a bit of rebuild on some other projects to make better use of my Micro SD cards. The starting point was a 8 GB card with just a basic Raspbian set up on it.

So, for starters, I found that the one I have in my recently set up music station Raspberry Pi is a whopping 128gb. Contrary to what one might thing, I don’t need a 128gb card in my music station, the music is stored on the NAS over the network. It also has some old residual projects on it that should really be cleaned out.

So stuck the 8GB card in that device and did the minor set up needed for the music station. Specifically, configure VLC for Remote Control over the network, then add the network share. Once I plugged it back into my little mixer and verified I could remote play music, I moved on.

This ended up being an unrelated side project though, because I had been planning on getting a large, speedy, Micro SD card to stick in my Retroid Pocket. So I stuck that 128GB card in, the Retroid and formatted it. This freed up a smaller, 32GB card.

I also have a 64GB that is basically not being used in my PiGrrl Project I decided to recover back for use. The project was fun, but the Retroid does the same thing 1000x better. So now it’s mostly just a display piece on a shelf. Literally an overpriced paperweight. I don’t want to lose the PiGrrl configuration though, because it’s been programmed up to work with the small display and IO Control Inputs. So I imaged that card off.

In the end though, I didn’t end up needing those Micro SD cards though, I opted for an alternative option to replace the Pi, with Docker on my secondary PC. I’ve been meaning to better learn Docker, though I still find it to be a weird and obtuse bit of software. There are a handful of things I care about restoring that I used the Pi for.

  • Youtube DL – There seem to be quite a few nice Web Interfaces for this that will work much better than my old custom system.
  • WordPress Blog Archives – I have exported data files from this but I would like to have it as a WordPress Instance again
  • FreshRSS – My RSS Reader. I already miss my daily news feeds.

YoutubeDL was simple, they provided a nice basic command sequence to get things working.

The others were a bit trickier. Because the old set up died unexpectedly, The data isn’t easily exported for import, which means digging out and recovering off of the raw database files. This isn’t the first time this has happened, but its a lot bigger pain, which isn’t helped by not being entirely confident in how to manipulate Docker.

I still have not gotten the WordPress archive working actually. I was getting “Connection Reset” errors and now I am getting “Cannot establish Database connection” issues. It may be for nothing after the troubles I have had dealing with recovering FreshRSS.

I have gotten FreshRSS fixed though. Getting it running in Docker was easy peasy. Getting my data back, was… considerably less so. It’s been plaguing me now when I try to fix it for a few weeks now, but I have a solution. It’s not the BEST solution, but it’s… a solution. So, the core thing I needed were the feeds themselves. Lesson learned I suppose, but I’m going to find a way to automate a regular dump of the feeds once everything is reloaded. I don’t need or care about favorited articles or the articles contents. These were stored in a MySQL database. MySQL, specifically seems to be what was corrupted and crashed out on the old Pi/Instance because I get a failed message on boot and i can’t get it to reinstall or load anymore.

Well, more, I am pretty sure the root cause is the SD card died, but it affected the DB files.

My struggle now, is recovering data from these raw files. I’ve actually done this before on a surver crash years ago, but this round has lead to many many hurdles. One, 90% of the results looking up how to do it are littered with unhelpful replies about using a proper SQL dump instead. If I could open MySQL, I sure as hell would so that. Another issue seems to be that the SQL server running on the Pi was woefully out of date, so there have been file compatibility problems.

There is also the issue that the data may just flat out BE CORRUPTED.

So I’ve spun up and tried to manually move the data to probably a dozen instances of MySQL and MariaDB of various versions, on Pis, in Docker, on WSL, in a Linux install. Nothing, and I mean NOTHING has worked.

I did get the raw data pulled out though.

So I’ve been brute forcing a fix. Opening the .ibd file in a text editor gives a really ugly chuck of funny characters. But, strewn throughout this, is a bunch of URLs for feeds and websites and well, mostly that. i did an open “Replace” in Notepad++ that stripped out a lot of the characters. Then I opened up Pycharm, I did a find and replace with blanks on a ton of other ugly characters. Then I write up this wuick and dirty Python Script:

# Control F in Notepad++, replace, extended mode "\x00"
# Replace "   " with " "
# replace "https:" with " https:"
# rename to fresh.txt

## Debug and skip asking each time
file = "fresh.txt"
## Open and read the Log File supploed
with open(file, encoding="UTF-8") as logfile:
    log = logfile.read()

datasplit = log.split(" ")
links = []

for each in datasplit:
    if "http" in each:
        links.append(each)

with open("output.txt", mode="w", encoding="UTF-8") as writefile:
    for i in links:
        writefile.write(i+"\n")

Which splits everything up into an array, and skims through the array for anything with “http” in it, to pull out anything that is a URL. This has left me with a text file that is full of duplicates and has regular URLs next to Feed URLS, though not in EVERY case because that would be too damn easy. I could probably add a bunch of conditionals to the script to sort out anything with the word “feed” “rss”, “atom” or “xml” and get a lot of the cruft removed, but Fresh RSS does not seem to have a way to bulk import a text list, so I still get to manually cut and paste each URL in and resort everything into categories.

It’s tedious, but it’s mindless, and it will get done.

Afterwards I will need to reset up my WordPress Autoposter script for those little news digests I’ve been sharing that no one cares about.

Slight update, I added some filtering ans sorting to the code:

# Control F in Notepad++, replace, extended mode "\x00"
# Replace "   " with " "
# replace "https:" with " https:"
# rename to fresh.txt


## Debug and skip asking each time
file = "fresh.txt"
## Open and read the Log File supploed
with open(file, encoding="UTF-8") as logfile:
    log = logfile.read()

datasplit = log.split(" ")
links = []

for each in datasplit:
    if "http" in each:
        if "feed" in each or "rss" in each or "default" in each or "atom" in each or "xml" in each:
            if each not in links:
                links.append(each[each.find("http"):])

links.sort()

with open("output.txt", mode="w", encoding="UTF-8") as writefile:
    for i in links:
        writefile.write(i+"\n")
Ramen Junkie
Ramen Junkie

Josh Miller aka “Ramen Junkie”.  I write about my various hobbies here.  Mostly coding, photography, and music.  Sometimes I just write about life in general.  I also post sometimes about toy collecting and video games at Lameazoid.com.

www.bloggingintensifies.com
Posted in: Linux & Open Source Tagged: #100DaysToOffload, Docker, FreshRSS, Projects, Raspberry Pi

Hard Drive Woes Part 2

April 13, 2023

This post is a follow up to my previous Dead Hard Drive post.

I used to hassle with PC hardware a LOT more than I currently do. I’ve kind of worked my way out of that gig honestly. I am at a point where I can afford shit for starters, mostly, so I’m not trying to cobble together workable machines from random parts. I also got tired of doing tech support for people, so I basically just, sort of hide that I can, because when people find out you can “fix computers”, now you’re vacuuming out 50 years of dust from a Pentium 1 in your backyard for a neighbor who refuses to just buy literally any cheapest machine at Wal-Mart for an infinite performance boost.

“Back in my day!” (fist shaking), you could pretty much just slap any drive with an Operating system in any machine and it would boot. Sometimes it would boot into an ugly driverless environment because it was ripped from another machine, but that was fixable. Things seem more complicated these days. I’m not blaming UEFI, and all that more secure BIOS stuff, but it’s a likely culprit. I think that better security is good, it just, is also part of why I can’t more conveniently fix my damn PC.

I say Conveniently, because that’s the core issue. I can still EASILY do it. It’s just… not convenient.

Shortly after messing with Linux a bit for troubleshooting, I did a bit of set up to use it as the main driver but, decided to just go back to Windows. I downloaded a fresh recovery image, sliced the Linux partition down to 500GB and reinstalled Windows.

I like Linux. I use Linux, almost daily, if not daily. It’s great for automation tasks and running server software and all that. It, kind of really sucks as a desktop OS. Don’t get me wrong, it’s usable, especially for simpler needs (literally anything not Gaming or Video/Photo Editing). I have run Linux as the sole OS on many machines, mostly laptops, and lots of Pis and Servers. I’ve used Linux off and on for over 20 years now. The problem here is, the main use case for my “Kick ass gaming rig” is well, gaming. Half the games I had slated “to play” from Steam are not available in Linux. I set up Hero Launcher for GOG and Epic, but like, my cloud saves didn’t work, and Fortnite doesn’t work and the whole thing felt a little off. Graphics also felt a little off, even though I did switch to using the official proprietary NVidia drivers.

Anyway, I went back to Windows. I spent an eternity downloading drivers and doftware and getting things set up properly. Unfortunately, the secondary drive I was now using as my primary, is just too slow to handle the needs of a lot of games as well. I had to roll Fortnite back to DirectX 11 for example, because it would take like 10 minutes to drop into a match because it would load shaders or some shit. For anyone not aware of how Fortnite works, it’s online, in an arena of players. If you drop in 10 minutes late, your character will have already landed in the map and probably be dead or dying.

So I bit the bullet and bought a new NVME drive. I planned to eventually, I just, did it sooner.

I went and downloaded Clonezilla to just mirror the Hard Drive to the NVME drive, which worked, but things would not boot.

There are plenty of possible solutions online, with recovery mode. I tried a few of them. But in the end, I have opted to just, reinstall Windows, again.

Which means redownloading drivers and shit… again….

I might be able to pull the Steam Downloads over before wiping the secondary drive, but I am not sure Epic will let me do that. Unfortunately, the larger games are from Epic, with Fortnite, Death Stranding and Final Fantasy 7R in that list.

It’s all, very easy.

It’s just all, very inconvenient.

Also, just because, and maybe for future reference, the install needs:

  • Network Driver – For some reason it doesn’t work on the generic.
  • TUF Gaming Amoury Crate – The motherboard seems to load this, and it find and installs all the drivers, which is nice, despite the cheesy name.
  • Windows Update
  • Color Scheme to Dark, no transparency
  • Firefox – Browser of choice, then log into sync and let it pull all my stuff in.
  • Steam
  • Epic
  • Visual Studio Code
  • Change One Drive settings to not sync everything but only some things.
  • Log into the Microsoft Account so One Drive and Office work, since no network driver means local account log in only at first
  • Share X – For Screenshots to folders
  • Display Fusion – For rotating desktop wallpaper
  • Synergy KVM – So I can connect to my other PC\
  • EVGA Flow Control – For the cooler
  • Remove all the cruft from the start menu, remove the apps list and recent files
  • Add a dozen network drives to File Exporer
  • Discord
  • Firestorm Viewer
Ramen Junkie
Ramen Junkie

Josh Miller aka “Ramen Junkie”.  I write about my various hobbies here.  Mostly coding, photography, and music.  Sometimes I just write about life in general.  I also post sometimes about toy collecting and video games at Lameazoid.com.

www.bloggingintensifies.com
Posted in: PC Hardware Tagged: #100DaysToOffload, Hard Drive, PC, technology

A Dead Hard Drive

April 3, 2023

I came down last night to drop some stuff off in the basement and shut the curtains, and sat down to check on something at my desktop PC, I don’t even remember what, and was slightly surprised to see that it was sitting at the BIOS Screen and not the Windows lock screen. My first assumption was, it did an update or something, and the cat was sitting on the keyboard, and cause it to enter the BIOS. They don’t usually sit on the keyboard, but it’s possible.

I rebooted the PC, and…. it just loaded the BIOS again.

Clearly something more than a cat issue.

Both the 1TB M2 NVMe drive and the 2TB add on drive were showing in the BIOS menu, but no boot options were showing available. In fact, it even specifically said something along the lines of “No boot options.” I tried resetting the BIOS settings back to factory default, I had toggled a few things so I could do virtualization, and it was no help.

I dug out a USB key with Linux and booted to that. I mostly wanted to see if I could access the drive still at all. This had to be done with an extremely weird and annoying bright yellow screen where everything was washed out. The Live OS would boot fine and look fine, until it actually got to the point of letting me do anything, when it suddenly seemed to give up it’s video driver causing everything to go wonky.

I managed to squint my way through it and the drive shows up, but it’s not accessible at all.

So I swapped out Ubuntu for a Windows Recovery USB Key. The recovery options (restore, recovery, etc) all failed. These gave a bit more information that the drive was “locked”. I tried a few more options at the command line that I found,

  • bootrec /fixMBR
  • bootrec /fixBoot
  • bootrec /rebuildBCD

But none of these changed anything. I could probably download and run a Windows ISO, but for the moment, I’ve decided on a different route. I booted back into Ubuntu, and just installed that on my 2TD spare drive. It would not take on the 1TB NVMe drive, and the 2TB secondary drive was just all games anyway, so nothing of value would be lost by wiping it clean.

I might, MIGHT just try running this way for a while, though it does have some disadvantages. Mostly, games. Almost everything I’ve been actively playing lately was through the Epic store. And a lot of the games I planned to get to in Steam, don’t work in Linux. There are ways to get them to work though, which I want to look into, but I have not had time yet. I do know Fortnite is flat out not going to work. Not a huge loss, I am kind of getting tired of it again anyway. It has some strict Anti-Cheat which won’t run in any sort of emulated environment.

I also still have my old desktop too I can use. So well, Fortnite may not be out completely, it just, won’t run quite as nice. In fact, I can probably run most of the stuff I want on that machine that won’t work directly in Ubuntu.

Another thing worth mentioning, I am not really out anything file wise. A handful of downloaded files for “TODO” projects that I could download again. I basically never work with files directly on any particular system anymore, it’s always files on the NAS or files in One Drive. The only thing I really lost were the handful of custom Stable Diffusion Embeddings I had created, and I have been meaning to try to rebuild better versions of those anyway.

It will be interesting to see how performance is compared to Windows though. This PC is a pre built gaming PC, so I am sure it’s been somewhat optimized for use with Windows. I have not had a chance to really test it out in a Linux environment at all yet, but I’m interested to see the results. I’d already been toying with the idea of running Linux on this machine but I was worried about how it would handle things like the water cooler. I already don’t have the ability to control the lights on my Keyboard and Mouse, but there may be software to do that available if I look into it.

All in all, I am irritated that the drive died, but I’ve taken it much more in stride than one might expect. I will probably poke at the Windows system some more as well though. The drive doesn’t really act like it’s dead, more like, it’s got some sort of software glitch going on.

Ramen Junkie
Ramen Junkie

Josh Miller aka “Ramen Junkie”.  I write about my various hobbies here.  Mostly coding, photography, and music.  Sometimes I just write about life in general.  I also post sometimes about toy collecting and video games at Lameazoid.com.

www.bloggingintensifies.com
Posted in: PC Hardware Tagged: #100DaysToOffload, Hard Drive, My PC, PC
1 2 … 16 Next »
Status Posts
Code Projects
Photo Gallery
Music
About
What I Use
Series Articles

Categories

Subscribe via RSS

RSS feed RSS - Posts

Social Media

MastodonLinkedIn

emailInstagramInstagram

GitHubLetterboxdDuolongo
GoodreadsLast.fmElite Dangerous INARA
Lameazoid Logo

Support this site.

Join EFF!

Copyright © 2023 [Blogging Intensifies].

Me WordPress Theme by themehall.com