EXTENDED STATS BLOG

Working on the User page is a bit of a pain in the backside.  For those of you who are new here, let me explain. After you create an account and log in, the Log Out button appears with your (Extended Stats) user name underneath it. This works extremely unreliably! I’m *still* debugging that.

If you click on that user name, you get taken to the user page, which contains the user widget, which is the bit that is supposed to let you choose your BGG user name and make geek buddy lists. It also shows you the information that Extended Stats stores about you, which actually does work, even it looks like a lot of waffle (so far).

Now I spent quite a bit of time before I got addicted to Assassin’s Creed: Origins making the user widget work. In particular I built an Express server and coded the autocomplete functionality in that and deployed it with Elastic Beanstalk and then undeployed it from Elastic Beanstalk and just stuck it on an EC2… and now Chrome says that as the user page was loaded using HTTPS (secure), it can’t call the Express server on the EC2 using HTTP (insecure) as that would break the security. It is correct to do that. So today I needed to add a security certificate to the EC2, but with AWS you just can’t do that. There is no way.

So what I figured out I could do is I could configure api.drfriendless.com, which does have HTTPS, to direct calls to the autocomplete code on the EC2. And then I need to change the user widget to use https://api.drfriendless.com instead of http://eb.drfriendless.com.

As if all of this isn’t a complete pain in the bum enough, extstats.drfriendless.com is served by the CDN, Cloudfront, so it caches all of its resources. I am supposed to be able to tell it to dump particular resources from the cache, but that hasn’t been working for me. It used to, but now I must be doin’ it wrong. So after I change all of those things, I have to wait a day for the changes to take effect. This is a seriously painful development process.

Of course, I need a test server to try this out on. I do have test.drfriendless.com. The test server doesn’t use CloudFront, which means that it doesn’t have HTTPS so it won’t have the problem anyway. But also, because it doesn’t have HTTPS, auth0 (which manages the logins and passwords) won’t talk to it. So in this particular case, the test server cannot possibly work.

Ah well, enough bitching. I got two new patrons yesterday, for a total of 4. Four wonderful patrons! Muahahahahaa! At this rate I’ll be rich about when I get the user page working properly.

 

This Christmas season, I am being, as usual, as unChristmassy as I possibly can. Quite apart from work never actually stopping for me, and this being a busy time of year, I’m a grumpy old man. However as this blog is not about my misanthropy, let me tell you about the code I’ve been writing.

On December 1 (oh my goodness, so long ago) I showed off the component I had got going which allowed you to enter a list of geeks. I also complained about Angular Material. Since then I’ve been working on the user data page, and maintaining the love-hate relationship with Angular Material. Mostly hate.

The user data page does two things – it allows you to edit data attached to your login on Extended Stats, and it tells you what that data is. The first is obvious, the second is my interpretation of a GDPR requirement. It might seem silly, but I kinda believe in what GDPR is trying to achieve and want to build accountability in from the ground up.

Now, editing the user data. The first thing you can edit is your BGG user name. As anyone can see the data on Extended Stats relating to your BGG user name, even without being logged in, you don’t need to fill that in. However my intention is that when you’re logged in, I’ll give you hyperlinks to the pages for that BGG user. And in fact, you can have multiple BGG user names, which will be convenient if you maintain stats on behalf of your spouse or your board game group.

Then there’s buddy groups. A few years ago I made this graph of new games played over time for my local gaming group. And I want so badly to be able to provide this graph to all of you!

So the plan is that this sort of graph will apply to a group of geek buddies. And to save you entering that group of geek buddies every time, I will store it in your account. And if you want to do it for your other gaming group as well, you can create a different buddy group.

Sadly, the work on the user page is still not complete. However most of the wrangling with Angular Material seems to be over, and I have to connect the results to the database. As the authentication system refuses to work with the test system, I need to test that out on the real system, which is kind of painful. So, work continues at its usual crawl.

I was away last weekend on Family Business, so I just let the site run, with the beanstalk going. The bill for November arrived, showing a dramatic increase in EC2 (virtual server) costs.

I’ve been very slack about posting about costs (because it’s so boring), so I made a (quite bad) chart of the components of the costs over time (the last 5 months). In July, the Cloudwatch  costs were high, because I was doing too much logging, and I fixed that. In September and October the Lambda costs were high, so I brought in Elastic Beanstalk to fix that. In November the EC2 costs were high because of the Elastic Beanstalk, so today I tried to fix that.

The reason Elastic Beanstalk is expensive is because it adds another server – so now I have two servers, one for Express and one for the blog – and that has a certain fixed cost. But then there’s another machine called the load balancer, which decides which of the Express servers will handle the request. In my scenario, the load balancer seems like an unnecessary luxury, given there is only one Express server. So at $8 for the month, that had to go. The virtual machine that Express runs on was more expensive than it needed to be, too, so I decided to make that an even smaller one. Those changes should save more than $10 / month.

Of course it wasn’t that easy. One of the nice things that Elastic Beanstalk does for you is create the server environment that the application runs in, so I had to redo that bit myself. So today I learn how to create an Application Machine Image,  how to deploy my Express application onto a server, and how to run an Express server with permissions to use port 80. The last bit was much harder than it deserved to be.

So apart from this recurring trauma, the site is going quite well.

The dithering of the updates has really evened out the load on the database, so now the database WHICH IS COSTING ME $30 / month looks like it might actually be able to do its job for a while. The AWS database service seems like extraordinarily poor value. On the other hand, it does backups and stuff for me.

The Lambda performance issues seem to be fixed for the moment as well.

It looks like it might be time to write some functionality.

So after arguing with Angular Material for most of the day, to the great disgust of my dog, I got this going:

It’s for editing your geek buddy list. Angular Material looks quite nice, when I can get it to work. I also quite like this idea of integrating demos into the blog posts. On the other hand, the demos are useless if I don’t then go put them into something on the site.

Well, after a week where everything was broken, I’ve made some progress. First of all, the big laptop got repaired and is better now. That means I could get the autocomplete demo from it, and then I was able to fix the bug in the web site deployment script (by abandoning the bit that mysteriously broke). That means the autocomplete demo is working, and here it is. Type someone’s geek name in:

That’s running against the Elastic Beanstalk server that I was waffling about a couple of weeks ago. So that’s all good. Next I have to remember why I was doing that.

Now I have also been working on the downloader performance, and there’s some good news about that too. I found a couple of places where I could do multiple database inserts in one transaction, and that has made a great difference. Here are some graphs of database performance over the last 2 weeks:

In the bottom right graph, notice that the green line has stopped bottoming out. That causes no more spikes in the bottom left graph, which is good because those are essentially site breakages. And the top two graphs show how the load on the database has spread out a bit.

This change in behaviour is also obvious in the Lambda performance graphs.

The nasty spikes in the top graph have gone, which is good because I think those are the ones which cost me money. In the bottom graph, the load seems to be continuing to flatten out, which is also a good thing.

OK, now I’ve solved those problems I really should get back to doing whatever the site was intended to do.

Well, I had a bit of a disaster during the week. On Thursday morning my laptop stopped charging, and now it’s in the shop being repaired for a hefty fee. It’ll be back some time during the week.

Luckily I have this other, smaller, laptop, so I can still do my work. It only took 7 hours to update it with all the work stuff… Well I guess that’s quicker than many other problems could have been resolved.

The bad news is that this laptop seems to be too underpowered to work on Extended Stats, which is not that big of a project, but it is a kinda complex one – it includes about a dozen Angular applications, and a couple of dozen Lambdas in a variety of locations, and I think the editor gets confused by the non-standard organisation.

Even worse, one of the bits of code that’s on the broken laptop is the autocomplete demo, which is one reasonably small thing that I could potentially have worked on on the smaller laptop. So that sucks as well.

Nevertheless, there is something to talk about. The work I did last weekend on putting the code in Elastic Beanstalk was aimed at preventing the really spiky bits in the Lambda performance graph (because they cost me money). Over the last couple of days we’ve been going through a period which should be spiky, and it has definitely changed.

By the way, these bits correspond to the update of all the users’ collections and played games, which happen every three days. I did two things.

First of all, I rewrote the bit that was appearing light blue in the bottom graph, because as explained in this post:

Bugs Bugs Bugs!

it was costing me a lot of money.

The second thing I did was to change “every 3 days” to “about every 3 days”. So when I schedule an update, I don’t schedule it for exactly 3 days, because all that does is preserve the periodic load. And as you can see, that flattened out the load a bit, and should continue to do so. I was calling this “dithering” to myself, but now checking on Wikipedia, dithering may be something more specific than that. Nevertheless, that plan is sort of working.

On the other hand, as Tevye would say, it didn’t seem to really help the database performance. The orange line is how hard the database is working, and making that change seemed to spread the peak out, but increase its total area. And then that caused the green line to hit the bottom for longer. And that’s bad. I’m hoping that with more dithering the green line might not get to the bottom at all.

I miss my big laptop. You don’t know what you’ve got till it’s gone.

As it’s Monday morning and I’m supposed to be going to work, this one will be short. I spent yesterday working on the autocomplete functionality, and after a great deal of mucking with Express security, I got it working. It looks lovely! However then when I went to put it on the site, the application I use to copy all of my HTML and so on up to AWS just refused to do anything. Computer says no.

That’s  the “fun” and frustration of this project. The tech is pretty much bleeding edge, so stuff just fails quite often. My test site, http://test.drfriendless.com, is completely broken at the moment!

I penetrated the impenetrable twaddle! After getting very frustrated last week with AWS security, I did some research, and finally found a description of a few things that I could understand. I now basically understand what a virtual private cloud is, and sort of how it works. That was really awesome news on Monday. Then on Tuesday we had some system problems at work and I got distracted sorting out other (AWS) stuff.

I was finally able to get back to the Express on Elastic Beanstalk project today. As I now understood the VPC stuff, I configured EB to use the VPC I already had. And then I commanded “Work now!” And behold, it did not.

I screwed around with it for a few hours, until I came to the conclusion that the reason it wasn’t working was because it thought it wasn’t working. No really, that makes sense.

Elastic Beanstalk is designed for implementing a web site (which looks like one computer) by using a bunch of computers in the background. If one of those bunch of computers goes bad for some reason, EB throws it out and replaces it with a new one. So to do this, EB has to know how to tell whether one of those computers is bad. And for a long time, I had that wrong.

However even after sorting that out, I still can’t get it to work. I’m beginning to suspect it’s a networking problem again – although Express “works” when I run it on my own machine, it doesn’t work when I connect to either the load balancer (the part of EB which farms work off to the worker machines) nor when I try to connect to the individual machines. My recollection from how we do this at work is that *should* work. So I’m back trying to figure out networking. And since I have known that stuff since Monday, I thought I got it…

I’ve also been considering whether I should actually get back to working on functionality (i.e. pretty things) instead of this AWS navel-gazing. But I should not. The bill for October (which I still have not blogged about) was a bit high, and I need to get this project working to cut down the costs. While it’s costing me too much, this project is at risk, so I need to get that stuff sorted.

The good news is that I was plagued by problems like this for about a year while I was trying to get this project off the ground, and with perseverance and experience I managed to figure them out. I feel like I”m making progress, even if it’s just developing a callus where I’m bashing my head on the wall.