I decided I needed a loading widget for the pages where a great deal of data needs to be loaded (i.e. all the useful ones). So after copying some stuff and fiddling with it, I came up with this:
I kinda like it. I like the colours. If I knew anything about graphic design I’d know how to proceed from that to make a colour scheme for the site.
To the uninitiated, developing a web site like this is just all beer and coding and living the high life from the Patreon proceeds. I have other responsibilities though! I’m fully conscious that the site looks a bit amateur, but that’s because I am indeed an amateur in many aspects of web development. I’ve been reading books about colour, and design – in the aesthetic sense, not in a how-to-do-it-in-HTML sense. So I’m hoping that over time some sense of style will infect my brain, and I’ll apply it to the site.
Of course the Plays page which I was touting last week failed miserably. It complains about the base setting being incorrect. I suppose it’s telling the truth, but it’s a bit annoying to find these things out in production. So I’ve deployed a new version and due to the CDN, you might see that new version tomorrow. In the meantime, you can play with the test site with URLs like this: http://test.drfriendless.com/plays.html?geeks=Friendless,karlsen,jmdsplotter which show off the colours and the multiple geeks.
Another thing I managed to sort of achieve today was compression. A graph such as the one above retrieves a lot of data from the database – maybe 2 megabytes. Sooner or later someone has to pay for that data transfer, and someone is gonna be me, so I wanted to cut that down. Luckily, web servers and web browsers have this agreement that the browser can ask for the data to be gzipped, and if the server can do it will. One of the things AWS’s API Gateway gives me for free (for which I forgive it for being a massive pain in the arse other times) is that I can just tell it so send data compressed, if the web browser asks for it.
So I changed my code to ask for the data to be compressed, and Angular (the JavaScript framework I use to write the pages) said “NO! You are not allowed to ask for that!” Apparently it’s a sin to be running around asking for stuff to be compressed willy-nilly. And the word on the street was that the browser does it automatically.
Which it plainly was not doing. So I had a think and a search and another think, and remembered that as the request for the data was coming from extstats.drfriendless.com and going to api.drfriendless.com – a totally different place – the two ends don’t trust each other. So extstats.drfriendless.com could only ask for compressed data if it first got permission from api.drfriendless.com to do so. This is a thing called CORS, but don’t Google it, it’s horrendously complicated annoying crap that just prevents things from working. For example, it prevents sites from stealing your bank logins and that sort of stuff. But after I configured CORS, the browser was allowed to ask for the data to be compressed, and the server was configured to do it, and hey presto 90% data compression.
Although the site doesn’t have a lot of the functionality I want on it, yet, I feel that mechanically it’s approaching stability enough that I can start inviting lots of people to use it. They’ll be disappointed I expect, but I will use their misery as motivation.
The old site annoys me more and more each time I look at it. During the week we had a technician come to convert our internet connection from “cable in the house connected to cable in the street” to “cable in the house connected to fibre in the street”. That may make the old site run a little faster, but probably not a whole lot as that site’s big problems are the database architecture and the wifi link from the machine to the home network. But I had to spend an hour or two of misery trying to understand networking and Ubuntu config things that are just not problems with the new site. I think I’ll be pleased when I can close that site down.
Last time I blogged I mentioned how frustrated I was with Vega. Yesterday, Saturday, I had a look at the problem and I still had no idea. So I posted a big description on reddit and went and walked the dog or something. When I got back, a useful person had replied, but I couldn’t get what they were telling me. After a bit more head scratching this morning, I finally figured it out and I got the plays page doing something.
Number of different games played over time, for Karlsen
I’m claiming it mostly wasn’t my fault! I thought that if I imported Vega it would import an appropriate set of sub-libraries. Not at all, it imported an inconsistent set of sub-libraries. So two versions of things I didn’t specify didn’t with each other. I am grumpy about that, but when it works Vega is so beautiful I’ll have to put up with it.
Not that this graph is particularly beautiful, yet. I probably should have used a colour other than black for the line. When I start adding more users, each line will be a different colour and rendered with a different symbol.
I’ve been working towards that goal for MONTHS. After I got this much going, I had a look at how to get the geek buddy lists from the user data into this page. Then I got stuck with another bug for way too long, because Angular doesn’t tell you the URL of the page you’re on… and it doesn’t even say that it does, it’s just that it looks very much like it might, and that confuses a great many people.
So I’m continuing work on adding multiple users to this page. It was very nice though to be able to add a news item to the news table in the database for the first time since September.
Sometimes it would be more fun to bang my head against a brick wall rather than be a programmer. However programming pays better and I don’t have much experience in headbanging. This is no time for a career change.
I managed to make some progress on the User page. That’s the one you get to after you log in and click on your user name. It seems that it can now store your BGG name and your buddy lists, and then display them again afterwards. It has taken WAY TOO LONG to get that working.
I then continued work on the Plays page – the one that uses the data about all the games you’ve played. I’m working on a chart to compare how many new games you’ve played when with the same for your buddies from a buddy list. I’ve had the data gathering working for a few weeks, and I have just finished the chart. Well, sort of.
The charting tool, Vega, is the most beautiful and argumentative contrary piece of shit software I’ve ever used. After spending a couple of hours getting the chart to work in the debugger, I had to address why it didn’t work in the page. My first idea was to upgrade some versions of some things.
That was a complete disaster. Vega has internal conflicts, so now my code can’t compile and won’t run. Using the latest versions of everything, it doesn’t work. Using the same versions of everything from this morning, it doesn’t work. I have turned it off and on again so many times… so now I’m bored and I’m going to do read a book instead. This happens every time with Vega, I seem to be missing something.
I hate banking and I hate bills. Not because it’s money going out – I can cope with that – it just gives me anxiety. I am forever doomed to be a programmer (or possibly a novelist) because I hate thinking about money and I hate doing banking. This extends even to the email from AWS about the bills giving me anxiety. That’s why I had 3 unread ones in my mailbox until today, and they are issued monthly.
Despite my personal issues, the bills were great news, as illustrated in drop in the graph above. The database continues to cost a fixed amount (variations are due to varying numbers of days in the month), and the other costs are all dropping. I could probably do even more about the EC2 costs now that I have the lambdas sorted, but I feel that I should get some functionality working on the site, for a change.
