Category: Procrastination

So this is not a good news post (the sort that says “I added features”) but this is not a bad news post (the sort that says “I’m giving up computing and going to live on a remote island with Aishwarya Rai”), it’s a just a post to tell you that despite all the evidence, I’m not dead and Extended Stats is still on my mind.

I’ve been trying to solve a problem connecting two bits of AWS together. Two years ago I would not have understood the problem, so I will not try to explain it to you. In fact two years ago I had a similar problem that depressed me so much that I stopped working on Extended Stats Serverless for about 4 months – that was before I announced anything, and I was doing a proof of concept, and I could not prove the concept. So I knew that it might be difficult, but really if I want to do AWS development I have to be able to work through these things.

Anyway, the good news is that with a lot of research (i.e. even checked sites other than StackOverflow, and I read documentation), and a lot of daydreaming, I have fixed this particular problem.

Daydreaming is an important technique in computer programming. When something doesn’t work, and it damn well should, my approach is to do something else and dream about what stupid thing might be happening. So I dream about how one bit probably works, and what its concerns are, and how the other bit probably works, and what its concerns are, and what might be missing in the middle. I find it’s best to do this in the shower, so all development offices should have showers :-). However today I figured the problem out while cooking breakfast. I wanted to watch Peaky Blinders, but once I thought of what might be going on I had to turn off the TV and try it out!

Anyway, that’s good news. I am very conscious that the old site is still down (I need a case fan. There is one computer store I know of that I can drive to, so after dropping my wife at work on Wednesday I went there, and they didn’t open till 10am. It was 9:30am and I’d already spent enough time not doing what my boss was paying me to do. I gave up and ordered a fan on-line.) and I need to address that. The new site is all people have at the moment, and it still doesn’t have the content people want.

This is why I need to move the site into the cloud! The fan in my server has gone out of balance or whatever they do, and started making a weird grinding noise. My wife and my niece, who both work in that study, complained about the noise. I had to shut the machine down, because if the fan dies in the middle of the night and the machine overheats, everything is dead, which would be bad.

Come to think of it, that study is not air-conditioned, and I’m expecting summer to be hotter than ever. I’m not sure the machine will be healthy. I might have to consider doing a lift and shift, and putting the old site into the cloud as well. As long as I use a local database (on the virtual machine, not managed by AWS) it will only be a little expensive. Hmm… I’ll go have a shower and think about that…

I have been a bit slack with this project recently, but I’m going to blame most of that on playing games. I’m dungeon master in a D&D campaign, and I spend a lot of time researching things like “what’s the most explosive substance there is?” and “what are the middle names of the member of Destiny’s Child?”, even when I’m not at the gaming table being way too extroverted.

However I have indeed also been working on something for the stats site. One of my intentions is that the data presented on the site be customisable – if you don’t want to count expansions, then you shouldn’t have to. To this end, all of the data presentation pages take a selector which tells them which data to present. In case you have never heard of selectors before, they’re a formula which specify which set of games you’re talking about. The selector for the Favourites page is “all(played(ME), rated(ME))“, which means “games which have been played by me and rated by me”. Now you might very reasonably want to further restrict to games that you own, or to exclude books, or to only look at children’s games. So ideally you would be able to edit the selector.

On the Favourites page you used to be able to, but it looks like I recently broke it, and the interface to do it was pretty hard to use. So I am working on making a new one. That comes in two parts.

The page itself will have a basic selector which makes the page work. The Favourites page can be run against games you haven’t played, but it makes for some pretty bad data. In fact the page can suggest a variety of different selectors that I think are useful. These are presented in a dropdown list box. Furthermore, you can type into the list box, either to find an existing entry or to add your own entry. And of the entries in the list box, you will be able to mark some as your favourites.

No such UI component like that exists, so I wrote one. I just got all (well, most) of the finicky details working this morning, and put the code on GitHub here: https://github.com/DrFriendless/favcombo . It looks like this:

When you come to the page, the options in the list box will be the suggestions from the page plus any that you previously marked as a favourite. So if you added a selector that you liked, it will be stored for you. Note to self: this will only work if you are logged in, in fact it’s cruel to let you favourite things if you’re not logged in.

But to make that work, I need to do some work on the storage and retrieval of personal data for logged-in users. I haven’t got to that problem yet, I guess it’s what I’m looking at for the rest of this weekend!

There’s also the issue that the site logs you out too soon – it annoys me so much that I don’t generally bother to log in, and it’s my site!

Anyway, as you see things are happening bit by bit. I’m not dead yet. You can’t kill the DM.

My wife has leave at the moment, so she suggested that we go on a holiday interstate. I do not have leave at the moment, but I’m the one who does the long-distance driving so I was tangled up in this plan. I often work from home so my boss said it was OK, so next thing you know I’ve driven a thousand kilometres and am staying at a holiday unit at the beach. Here is the view from the balcony.

Of course, I am not on holiday, so I’m busy coding away in between dropping the family at the shops and the train station and so on. I have a couple of pretty big deadlines so I really am busy.

Even worse, the internet here is not as good as at home. For work stuff that’s generally OK. However I’ve done a couple of updates to extstats.drfriendless.com, and it takes quite a while to upload 50 megabytes. However I did finish an update to the user page (the one you get to if you click on your name when you are logged in). It still looks pretty bad because I can’t work Angular Material very well yet, but I think it’s more useable. I’ve also been trying to start using some of the configuration you can set on that page. However I have not really had the time to make a lot of progress. Worst holiday ever.

On the other hand, I was watching my wife web-surf last night, and discovered that the h-index is a popular statistic amongst academics. An academic’s h-index is the highest number n such that they have n papers which have been cited n times each. And of course being academics, whose performance is measured by their h-index and similarly absurdly trivial metrics, they think way too much about this sort of thing.

For example, they have a g-index. The g-index is the largest number n such that their n most-cited papers have been cited on average n times each. We don’t have that metric. NOT YET!

They also have a rational h-index, which is approximately the h-index but with some indication of how close you are to getting to the next number. So we definitely want that! The formula (which took a while to track down) is:

• say your h-index is h
• and the minimum number of plays you could play to get it to h+1 is n
• then your rational h-index h_r is:

h_r = h + 1 – n / (2h + 1)

and of course you keep the fractional part. OMG, I am so excited! I can’t wait to implement it! But right now, I have to have a “holiday”.

“A Man, a Plan, a Canal – Panama”

The lovely palindrome suggests that the building of the Panama Canal was planned and executed in a rather neat and reversible way. So when I consider my development process which is kinda haphazard, I feel a bit inadequate and envious. Fortunately, a quick glance at the Wikipedia entry on the building of the Panama Canal suggests that its 33 year construction was not without drama, difficulty or incompetence. Ferdinand de Lesseps, the man with the plan, was sentenced to go to jail for his role, but never actually did.

So that makes me feel better. The first Extended Stats site started as a few Python scripts. I spent years running them nightly and uploading them to free hosting sites, which were universally shitty, resulting in my decision to host the site at home. Maybe only a year after that I rewrote the scripts as a proper web site. Between the shitty free hosting and the world’s mistrust of dynamic DNS, the site name kept changing. As web users continue to expect more (e.g. https) and I become more professional in my approach, it continues to evolve.

The same thing is happening with the new site. I spent a good deal of 2015 and 2016 thinking “I should make a new site”, then a good deal of 2017 thinking “hmm, what would be the best way?” And then I spent a few more months trying to get those ideas to work. It was only in June 2018 that the technology started to work for me. About then I realised that Node.js, which I had chosen to write the server-side in, was not one of my talents. So I wrote some fairly bad Node code for a while, and maybe I am still doing so. Though it’s a few months since I’ve found a way I like better.

Similar things happened with the web pages. Although I am competent with Angular, when I started I didn’t know what I wanted on the pages, and I still don’t, really. However they are slowly converging. Whenever I write a new page there are some bits I like and some bits that are horrible. When I’m done, I take the bits that I like and put them into a library on NPM, and use them again later. That makes it easier to re-use those ideas on later pages, or on revisions of existing pages.

So this morning I sat down, determined to put the Plays Of Games Owned table into the new site. The first task was to find the right place for it, which would be on an existing page which already has the correct data. I looked at the Owned Games page, and noticed that it was the first page where I had implemented the pattern of the page loading a set of data, and multiple components rendering their views on it. However some code from that page had been put into a library, so I fixed up the page to use the library and stuck in a loader component while I was there (the coloured bobbles that indicate that the data hasn’t arrived yet).

However the Owned Games page doesn’t have plays data, so it’s not the right page for that table. So I next considered the Favourites table, which does include plays. The selector isn’t quite right though, as the Favourites table defaults to games played and rated, and isn’t anything to do with owning them.

The Favourites page one of the first pages to use Vega charts, so it had a Charts button that you could click on to get to the charts. That preceded the way I do it now where each chart appears on the page as its own thing, and uses the data the page provides. So I moved those charts out onto the main Favourites page, and wired them up to receive the data. Then I noticed that the Favourites page was the one where you can change the selector, but the new selector was injected into the table, rather than into the page. So I went back to the library and changed the way the config component worked, so now you can change the selector on the Favourites page, and the table and all of the charts update. It’s still clunky, but it works. Oh, and the Favourites page had one of the old-style tables without the handy header bar, so I fixed that up as well.

And that’s why it’s now late afternoon and I have not yet achieved the first thing I set out to do today. The site has evolved a little, and I look forward to wiring the selector changer into other pages, and hopefully making it nicer when I do so. I’ll see if I get to try again tomorrow.

“A man, no plan, a clusterfuck, a clean up.”

Before I fall asleep, let me bash out a quick blog post so that you know I’m not dead. Instead I have been occupied with visiting family, which is the next best thing.

I have been doing some work on the site, but it hasn’t turned into useful functionality yet. I believe I’ve got the API for retrieving plays of games working, so next I have to produce a user interface which displays some of that data. I did a little bit of work on that this evening before I had to drive somebody somewhere.

I’ve also been chatting with the guys from https://www.rollforgroup.com. There are more of them than there are of me! I met the head honcho there at a MongoDB meetup before I started this rewrite, and when you find another programmer obsessed with board games you’ve got to be friends with them, right?

I also got a couple of new Patreon patrons early this year, which was wonderful! But that was exactly at the time that I went away for a week to play board games (including Gloomhaven, for the first time), and then when I got back the Family Business started. Now I remember why it took so many years for the old site to evolve to its current state.