Python

Ending My Experiment with Flask

I really like Python, but I have one major gripe with it. It’s a pain in the ass to actually share any projects. Python can’t easily be packaged into an executable, and it doesn’t really work well on the web. I had some vague high hopes of building some Flask based web apps, and I may revisit it at some point, but even just trying to share some of the projects I made during my Python class had become a cumbersome pain to manage.

I had originally set up the blog project over on JoshMiller.net. A domain I have, and don’t really want to dump, but don’t really have a use for. I also do not need another blog. “Traditionally” I would use that domain for “personal blogging” but that’s what this site is for, and I’m happy with the state of things here. The Flask blog was also extremely basic and would require a lot of code work to make workable, with things like image management and an RSS feed and some sort of theme system etc. I could probably do it, but…. why?

One thing I’ve learned after working on many fruitless projects over the years. It’s ok for a project to be kind of pointless, but it’s not ok for it to just be completely pointless. There are plenty of much better platforms for blogging, and frankly, all of them are “Just use WordPress”. Another angle is learning how to run sites with something besides Apache, but that tine would be better just learning Nginx.

Anyway, part of the point was to take the base Flask Blog app, and integrate other Flash Apps into it. The problem is, all of the ones I have built are designed to be stand alone apps. They often reuse some name space so i can’t just copy/paste the code in. Then things start to get screwy with modules and imports. The code itself is also not super reusable. Flask can host HTML code, but it’s all very specifically laid out and structured. I can’t just drop the same code into a folder somewhere and run it on a straight Apache instance.

What does work well for this, is JavaScript. Which is basically entirely intended for web based use. Most of the ideas I have had for projects would be much better suited running as JavaScript sites that are much more portable.

Anyway, all of this is mostly to say, I have disabled the Flask Blog I had hosted on JoshMiller.net and rolled it back to the broken landing page I was building. I don’t really care about this site anyway. Hell even the landing page concept is redundant because I’ve got a nice little Github Page serving that purpose.

Code Project: Fresh RSS to WordPress Digest

I actually briefly mentioned this project when I write about moving from TinyTinyRSS to FreshRSS. This has become a bit of an evolving and ongoing project however, so I’ve decided to catalogue it in it’s own page. This little script worked out much better than I expected, and I’ve modified it a bit over time, and have ideas to modify it going forward even more. Starting off, the code can be found here in this Github GIST.

I’ve left a bit of commented out code that i might use later for troubleshooting or adding additional features. The general gist of the code, it pulls the last 24 hours worth of news stories I have favorited from my FreshRSS install, then formats them into a digest format and posts it here, in this blog. They get sorted into their own category, you can find them here.

This is basically a thing I’ve seen others do that I’ve wanted to do for a while. It’s also partially just for my reference more than anything, it’s sort of a log of everything I have found interesting on a particular day more than anything. Others may or may not find it interest, which is why I also filter that category out of the home page feed.

Originally, it was just a list of URLs and titles. I realized that it might be useful to have SOME idea what the link was about before clicking it, so I have been playing with the summary as well. My first attempt was a bit dodgy because it actually posted the entire article as the summary. Currently, it just arbitrarily chops it off at a few hundred characters. I want to improve it even farther at some point by pushing it through some summarizing AI and getting an actual proper summary but I have not gotten there yet.

There re a few other things I want to add but I’m not sure they re easily possible. Firstly, I would love to be able to parse some sort of categories into the digest. So say, all the “Video Game” links are together and Music links are together. FreshRSS has categories but they don’t seem to show up in the feed anywhere.

This would also allow me to split these posts between this blog and my other blog, Lameazoid. I do share interesting video games news from FreshRSS, but I mostly don’t share Toy related articles, because it feels a little TOO FAR out there for what I want to post to this blog. If there were a way to have the categories, I could easily have the script split the feed by categories and post a digest to each blog.

I also wish there was a way to add my own notes and commentary occasionally. I don’t think it showed up in the feed either, but TinyTinyRSS had a notes feature. I am not sure if FreshRSS has that as well. I probably should try to at least suggest these features to the creators on GitHub, or maybe get really adventurous and create my own plug-ins for FreshRSS to accomplish these tasks.

Code Project: Automated List From Reddit Comments

This is one of those quick and kind of dirty projects I’ve been meaning to do for a while. Basically, I wanted a script that would scrape all of the top level comments from a Reddit post and push them out to a list. Most commonly, to use on /r/AskReddit style threads like, well, for this example, “What is a song from the 90s that young people should listen to.”

Basically, threads that ask for useful opinions on list. Sometimes it’s lists of websites or something. Often it’s music. The script here is made for music but could be adjusted for any thread. Here is the script, I’ll touch on it a bit in more detail after.

## Create an APP for Secrets here:
## https://www.reddit.com/prefs/apps

import praw

## Thread to scrape goes here, replace the one below
url = "https://www.reddit.com/r/Music/comments/10c4ki0/name_one_90s_song_kids_born_after_2000_should_add/"

## Fill in API Information here
reddit = praw.Reddit(
    client_id="",
    client_secret= "",
    user_agent= "script by u/", # Your Username, not really required though
    redirect_uri= "http://localhost:8080",
)


submission = reddit.submission(url=url)
submission.comments.replace_more(limit=0)
submission.comment_limit = 1

for x in submission.comments:
    with open("output.txt", mode="a", encoding="UTF-8") as file:
        if "-" in x.body:
            file.write(str(x.body)+"\n")
            # print(x.body)

The script uses praw, Python Reddit API Wrapper. A Library made for use in Python and the Reddit API. It requires free keys which can be gotten here: https://www.reddit.com/prefs/apps. Just create an app, the Client ID is a jumble of letters under the name, the secret is labeled. User Agent can be whatever really, but it’s meant to be informative.

The thread URL also needs filled in.

The script then pulls the thread data and pulls the top level comments.

I’m interested in text file lists mostly, though for the sake of music based lists, if I used Spotify, I might combine it with the Spotify Playlist maker from my 100 Days of Python course. Like I said before though, this script is made for pulling music suggestions, with this but of code:

        if "-" in x.body:
            file.write(str(x.body)+"\n")
            # print(x.body)

It’s simple, but if the comment contains a dash, as in “Taylor Swift – Shake it Off” or “ACDC – Back in Black”, it writes it to the file. Otherwise it discards it. There is a chance it means discarding some submissions, but this isn’t precision work so I’m OK with that to filter out the chaff. If I were looking for URLs or something, I might look for “http” in the comment. I could also eliminate the “if” statement and just have it write all the comments to a file.

Advent of Code 2022, Day 14

Man, I really enjoyed today’s puzzle. Like, a lot. I think because it kind of felt like a game level, and probably also because it’s fluid dynamics and I am totally into Physics and Engineering shit.

For the “Plot” you enter into a cave and discover a cavern with sand falling from the ceiling. The sand accumulates in a pile and “flows” around based on some simple left then right rules. This problem consisted of a few separate but connected steps.

Step one, create an empty “cave”. This was simple enough, especially now that I remember how stupid lists are. Last time I needed to make a grid, I was appending a list and it turns out that Python doesn’t actually copy lists unless you explicitly ask it to. Which is frankly, “Fucking Stupid”. But whatever, list.copy() works too.

Step 2, draw the rocks from the input file. Each line consists of a start note, then a series of connected dots to the end point of a line of rocks.

Step 3, was to pour the sand. Which involves dropping a “chunk” of sand, down until it hits the floor, then flowing it left or right to fill an area. Once the sand starts falling off, then display the count of the total chunks. If I were more clever about my code, I could build a sweet little ASCII animation of each step, but I probably won’t anytime soon because well, I have other things I need to do too.

Part 2 modifies this, by adding a floor, instead of counting the amount of sand until it fills, and falls into the abyss below, now you count until it fills then fills all the way back to the top. This actually screwed me up a bit.

The coordinates given are all large, like, in the 500 range. In order to make my rock formation manageable, I had cut these down by the min max values so the cave was not much wider than the rock formation. The problem is, now I need to accumulate a pile across the floor, so I need the width. Like, a LOT of width. So I had to modify my code all over to bring the width back to my cave matrix.

The code works for Part 1 and Part 2 at once. Basically, it finishes Part 1, like normal, display the output count, and, just for fun, an ASCII image of the filled rocks, then, it just, starts a fresh, slightly modified loop. For the modified loop, the break for “falling off” is removed. Instead, it checks to see if it can move, and if it can’t, before placing the sand block, it verifies if it moved at all by comparing it’s position to the start position. If it hasn’t moved, it breaks the loop, prints the filled screen, and the sand count total.

import math

with open("Day14Input.txt") as file:
    data = file.read()

lines = data.split('\n')

def draw_cave(lx, ly):
    grid = []
    line = []
    floor = []
    for i in range(0,lx*2):
        line.append(".")
    for j in range(0,ly+2):
        grid.append(line.copy())
    for i in range(0, lx * 2):
        floor.append("#")
    grid.append(floor)
    return grid

def draw_rocks(rocks,cave):
    for rockline in rocks:
        for i in range(len(rockline)-1):
            startx = int(rockline[i][0])
            starty = int(rockline[i][1])
            endx = int(rockline[i+1][0])
            endy = int(rockline[i+1][1])
            if starty == endy:
                xrange = sorted([startx, endx])
                for horiz in range(xrange[0],xrange[1]+1):
                    cave[starty][horiz] = "#"
            if startx == endx:
                yrange = sorted([starty, endy])
                for vert in range(yrange[0], yrange[1]+1):
                    cave[vert][startx] = "#"
    return cave

def show_cave():
    for i in cave:
        print(" ".join(i))

smallest_x = 100000
smallest_y = 100000
largest_x = -1
largest_y = -1
rocks = []
for line in lines:
    sets = line.split(" -> ")
    r = []
    for n in sets:
        nsplit = n.split(",")
        if int(nsplit[0]) < smallest_x:
            smallest_x = int(nsplit[0])
        if int(nsplit[1]) < smallest_y:
            smallest_y = int(nsplit[1])
        if int(nsplit[0]) > largest_x:
            largest_x = int(nsplit[0])
        if int(nsplit[1]) > largest_y:
            largest_y = int(nsplit[1])
        r.append(n.split(","))
    rocks.append(r)

# print(f"{smallest_x} {largest_x} | {smallest_y} {largest_y}")
# print(rocks)

cave = draw_cave(largest_x,largest_y)
# show_cave()
rocky_cave = draw_rocks(rocks,cave)

sand_start = 500
rocky_cave[0][sand_start] = "+"

captured = True
sand_count = 0
while captured:
    sand_pos = [0,sand_start]

    sand_drop = True
    while sand_drop:
        if sand_pos[0] > len(rocky_cave)-3:
            captured = False
            sand_drop = False
        elif rocky_cave[sand_pos[0]+1][sand_pos[1]] == ".":
            sand_pos[0] += 1
        elif rocky_cave[sand_pos[0]+1][sand_pos[1]-1] == ".":
                sand_pos[0] += 1
                sand_pos[1] -= 1
        elif rocky_cave[sand_pos[0]+1][sand_pos[1]+1] == ".":
                sand_pos[0] += 1
                sand_pos[1] += 1
        else:
            sand_count+=1
            rocky_cave[sand_pos[0]][sand_pos[1]] = "O"
            sand_drop = False

    # show_cave()

print(sand_count)
show_cave()
# Part 1 = 728

#### RESUME FOR PART 2 #####
captured = True
while captured:
    sand_pos = [0,sand_start]

    sand_drop = True
    while sand_drop:
        if sand_pos[0] > len(rocky_cave)-1:
            captured = False
            sand_drop = False
        elif rocky_cave[sand_pos[0]+1][sand_pos[1]] == ".":
            sand_pos[0] += 1
        elif rocky_cave[sand_pos[0]+1][sand_pos[1]-1] == ".":
                sand_pos[0] += 1
                sand_pos[1] -= 1
        elif rocky_cave[sand_pos[0]+1][sand_pos[1]+1] == ".":
                sand_pos[0] += 1
                sand_pos[1] += 1
        else:
            if sand_pos == [0,sand_start]:
                captured = False
            else:
                rocky_cave[sand_pos[0]][sand_pos[1]] = "O"
            sand_count+=1
            sand_drop = False

print(sand_count)

show_cave()
# Part 2 = 27623

Advent of Code 2022, Day 13

Have I ever mentioned before how much I hate Recursion?

So, I feel like I mentioned on a previous Advent of Code post, half the problem of these puzzles is figuring out how to massage the input. Which I started to work with until I found the eval() function, which I don’t recall using before. But it basically will take something like a string that looks like a list, say, “[2,[4,5,6]]” and make it into a list of lists. Which solved the first problem I had.

Then there was the checks itself. The puzzle was a series of lists full of lists and integers, that needed to be sorted based on some criteria. It basically amounts to a series of if checks, if the data is a list or an int and what to do, if the lengths are the same or different, if one value is larger than the other. Except it can be lists in lists, hence the need for recursion. I felt like I was really close, but then scrapped that for a new approach. Then I got stuck again, then I realized I needed to loop through the internal list. Everything eventually lined up for part 1 nicely.

On to Part 2. Which basically amounts to, using the algorithm on the entire list, instead of just pairs.

I was actually a bit disappointed because I was sure I had come up with a clever way to solve this without the sorting algorithm. I left it in the code, but it’s not used. Here it is anyway.

## Not Used But This really feels like it should have worked.
def decoder(decode_list):
    new_list = []
    for each in decode_list:
        each = each.replace("[]", "0")
        each = each.replace("[", "")
        each = each.replace("]", "")
        each = each.replace(",", "")
        new_list.append(int(each[0]))
    new_list.sort()
    print(new_list)
    print((new_list.index(6)+1)*(new_list.index(2)+1))

The idea was, that I would flatten each list out to an integer, then sort the data that way. The target values needed were simply, 2 and 6. The initial sort put 2 and 6 at basically positions 2 and 6. because 2 is less than 200 when sorting. So it dawned on me, that 2 and 6, would always be the “first” at the “start of the 2s” and “start of the 6s”. So the I just added the first digit to the list, then sorted that, and took the first 2 and first 6. And it still didn’t work.

Figures it wouldn’t be that simple.

While contemplating using nested for loops, I decided to check some other solutions, and found something I had not used before, “cmp_to_key”. After reading up on it, it seems it can be used with the sort function to generate sorts based on a key value returned from a function.

So I modified my sort unction to use +1, -1, and 0 instead of True and False.

Aaaaand, it still didn’t sort, it wasn’t sorting. I had forgotten that I needed to use eval() on each entry, to turn it into Lists instead of Strings. My sort wasn’t working because it doesn’t work on Strings. So I corrected that and bam! Working.

from functools import cmp_to_key

with open("Day13Input.txt") as file:
    data = file.read()

split_data = data.split("\n\n")
decode_data = []
for line in data.replace("\n\n", "\n").split("\n"):
    decode_data.append(eval(line))

def compare_lists(left,right):
    # print(left)
    # print(right)

    if isinstance(left, int) and isinstance(right, int):
        # print(f"{left} {right}")
        if left < right:
            return 1
        elif left > right:
            return -1
        return 0

    if isinstance(left, list) and isinstance(right, list):
        for value in range(min(len(left), len(right))):
            check_value = (compare_lists(left[value], right[value]))
            # print(f"{left[value]} {right[value]}")
            # print(check_value)
            if check_value != 0:
                return check_value

        if len(left) < len(right):
            return 1
        elif len(left) > len(right):
            return -1
    if isinstance(left, int) and isinstance(right, list):
        return compare_lists([left], right)
    if isinstance(left, list) and isinstance(right, int):
        return compare_lists(left, [right])

    return 0

## Not Used But This really feels like it should have worked.
def decoder(decode_list):
    new_list = []
    for each in decode_list:
        each = each.replace("[]", "0")
        each = each.replace("[", "")
        each = each.replace("]", "")
        each = each.replace(",", "")
        new_list.append(int(each[0]))
    new_list.sort()
    print(new_list)
    print((new_list.index(6)+1)*(new_list.index(2)+1))

index_sum = 0
for i in range(len(split_data)-1):
    set1 = split_data[i].split("\n")[0]
    set2 = split_data[i].split("\n")[1]

    match = compare_lists(eval(set1), eval(set2))
    # print(match)

    if match == 1:
        index_sum += (i+1)


print(index_sum)
# 6373 Too high
# 5997 Too low
# 6187

decode_data.append([[2]])
decode_data.append([[6]])
#decoder(decode_data)
decode_data.sort(key=cmp_to_key(compare_lists), reverse=True)
# for each in decode_data:
#     print(each)
print((decode_data.index([[2]])+1)*(decode_data.index([[6]])+1))

# 23520