Categories
Uncategorized

A Dynamic Teleswitch Update – Successful Switch!

I’ve finally managed to switch electricity suppliers! It’s been a long journey, and I described some of the background in my previous post. In this post, I’m giving an update on my journey, I’ve written a guide on how to follow in my footsteps, and I’ve added a few more practical thoughts related to these meters.

My Journey

As I mentioned in my previous post, suppliers other than scottish power are only required to offer me single rate tariffs: meaning I’d be unable to take advantage of cheap night time electricity for my space and water heating. Even in light of this, my scottishpower tariff is so expensive that this still works out cheaper than staying with them. The company I ended up switching to was Octopus: simply because they were the cheapest “big name” supplier for a single-rate tariff with my ~5300kWh/year usage. This turned out to be a good choice, as they did an amazing job of supporting me with my switch.

I initiated the switch using MSE’s cheap energy club and then emailed them immediately to make sure they had all the relevant information about my meter (both of the MPANs) and check that they’ll only be charging a single standing charge. They initially told me they’d be doubling my standing charge on account of my dual MPANs, but I cited the “Energy Market Investigation (Restricted Meters) Order 2016” and I never heard anything about the notion of doubling my standing charge again.

Once I got talking to Lucas at Octopus, I was surprised that he was able to offer me a 2-rate tariff. I was delighted at this since it means I can save a lot more money than I was ever expecting! The switch took a little bit longer than it should have done because of problems with the MPANs not being switched at the same time and scottishpower objecting because of this, but I think lessons were learned and this is all ironed out now.

Unlike on scottishpower, my octopus account shows my two MPANs as if they’re separate meters:

My octopus dashboard

One MPAN works exactly as if it were an economy 7 meter, taking two meter readings (my day and night rate readings). The other MPAN also shows as being on the dual-rate tariff but I only submit one reading for it: my control reading, which gets charged on the night rate. I’m actually getting billed separate standing charges for each MPAN, but I was credited with a year’s worth of standing charges upfront so one standing charge is effectively zero. I haven’t been with them long enough yet to have received a bill, but when I submit meter readings my account balance gets reduced by the correct amount, so it seems like their billing is working fine! Since it’s the middle of summer, I haven’t used any control rate electricity though, so I can’t test that yet.

I’m really pleased with the outcome of this – I was never expecting to be able to escape scottishpower so this is a huge relief. I hope this information helps other people to make a choice with their restricted meters. Octopus have gone above and beyond their duty: all that Ofgem require of them is to offer me a single-rate tariff. I doubt it’s been worth their time and expenditure dealing with me, yet they’ve put me on a fair, dual-rate tariff which they had no obligation to do. The overhead of managing my meter probably outweighs any profit they’ll make from selling me electricity, but I think this is the fair thing to do: for suppliers to take the hit on the overhead associated with legacy infrastructure rather than this being passed to unwitting customers. I hope Ofgem make changes to reflect this in regulation and improve choice for us in the longer term.

Guide to Switching

This guide applies to anyone on exactly the same kind of meter as me: Scottish Power Comfort Plus White Meter. Please feel free to try it with any other restricted metering infrastructure and let me know how you get on! I know of someone else trying this with an SSE THTC meter now too.

  1. Bring your account up to date and in credit with Scottish Power. They may object to your switch if your account is in debt so submit a meter reading and, if you can, make a card payment to put your account comfortably in credit. This may be easier said than done in these times, but reducing the debt may still help if you’re able. Bear in mind that you might not get any credit balance refunded for up to 11 weeks from when you start your switch, so budget for being without the money for that time.
  2. Decide whether you want a single-rate or dual-rate tariff. If more than half of your electricity is on your night or control rate, a dual rate is probably preferable. Check your scottish power bill and you should have a summary of your annual usage. You can see mine at the end of this section, for example. My night+control rate summed together are 1892+2124=4016 – way higher than my day rate of 1348. So in my case, a dual rate tariff was definitely the right choice
  3. Decide which supplier you want to choose. Long story short, I’d recommend choosing Octopus. Admittedly, I do stand to gain if you use my referral code further down, but even if this weren’t the case, I still think they’re probably the easiest supplier to deal with if you have a restricted meter. Their rates are very competitive at the moment too. If you’ve decided you want a dual-rate tariff, as far as I’m aware the only option at the moment is Octopus. It might be worth comparing other suppliers’ economy 7 tariffs and asking them if they’ll supply your meter and with what charges, but be prepared for disappointment! If you’ve chosen a single rate tariff, you’re theoretically able to use any supplier, except for some smaller suppliers. You might want to compare suppliers using a tool like cheap energy club. Make sure to choose “show only big-name suppliers”, because this will exclude the ones who are too small to be required to take on your meter. Theoretically, your switch should be seamless but it’s likely you’ll have to contact them to confirm that you do indeed want their single rate tariff, and to make sure that they transfer both of your MPANs without charging the standing charge twice. Because of all of this, you might prefer to choose Octopus even if you want a single-rate tariff.
  4. If you’ve decided you want to switch to Octopus, it’s best to email them since a restricted meter switch will require some manual intervention on their end. Hopefully it should all be seamless for you though! Email hello@octopus.energy and ask for Lucas – he dealt with all the quirks of my meter and he’s probably now Octopus’ expert on restricted meters! Tell him Alec Wright sent you and quote my referral code: quiet-sky-905 – that way we’ll both get £50 of free credit! They’ll need your name and address to start the switch.
  5. Submit your initial meter readings with your new supplier as soon as they ask you for them, to maximise your savings
Annual usage example

Other Things to Consider

A few other things you might want to consider when selecting a supplier or tariff, or after you’ve switched:

  • If you’re asking for a dual-rate tariff, you’re relying on the willingness of the new supplier to support this. As far as I’m aware, no supplier is obliged to do so. If you don’t stand to gain much from a dual-rate tariff, you might find it easier just to just choose a single rate.
  • Consider how you might change your energy usage. If you’re on a Comfort Plus White meter tariff as I was previously, chances are you use most of your electricity at night since the rates strongly incentivise this. But maybe if you were on a single rate tariff you’d be able to reduce your consumption, and slightly offset the loss of a cheap night rate? For example, maybe you often find your storage heaters are wasting a lot of energy heating your home when you don’t need it? If so, perhaps you could stop using your storage heaters and switch on a plug-in heater as you need it instead. Or maybe you’re heating a full tank of hot water overnight then finding that you don’t need it all? Maybe stop running your immersion heater overnight and just hit “boost” when you need some water. While this is all generally bad advice if you’re able to get 2-rate electricity, you might decide that the hassle of securing a 2-rate tariff outweighs the savings. Before I found I could get a 2-rate tariff, I was planning to do all of these things and decommission my storage heaters!
  • Something I haven’t looked in to yet is whether the process of a meter change is easier after switching. I wouldn’t rely on this but once you’ve switched, it’s probably a good idea to discuss with your new supplier about the possibility of changing your meter to a more standard one. If you can do this, it will probably make switching easier next time. There’s a risk of this being difficult if you’re renting but it may be possible – especially if the replacement is also a smart meter, as I don’t think it’s possible for a landlord to object to a tenant getting a “dumb” meter being replaced with a smart meter.

Conclusion

All of this restricted metering nonsense has consumed a lot more of my time and energy than I’d expected. I think I’m about to wrap this project, but before I do I’m going to contact an organisation or two to try and “pass the baton” in terms of looking out for restricted metering customers. I hope what I’ve discovered helps someone else follow in my footsteps and raises more awareness of the struggles associated with these meters. If anyone else tries this approach, I’d be interested to hear how you get on!

Categories
Uncategorized

Dynamic Teleswitch Meters

For most people in the UK, switching electricity and gas suppliers is an easy process – and you can save yourself a lot of money by comparing your options regularly and considering switching. I’d recommend Money Saving Expert’s Cheap Energy Club as a very easy-to use tool for doing this.

However, for a few of us, the options are much more limited. When I moved into my current flat in Edinburgh, I had to deal with a Dynamic Teleswitch Meter, a form of “Restricted Metering Infrastructure” – as Ofgem refer to it. Essentially, the meter has two circuits with three associated rates: one circuit behaves in a similar way to economy-7, being active all the time but switching between a day rate and a cheaper night rate at fixed times of day. The other circuit is switched on and off at the behest of a radio teleswitch signal, and only supplies my storage heaters. The switching times of this aren’t guaranteed – it can be anywhere within a certain window, but must be at least a certain number of hours per night.

My “Comfort Plus White Meter”, plus right hand fuse box for storage heaters and left hand fuse box for everything else

I suspect this form of metering was introduced before the electricity market was changed such that it was possible to change suppliers and so this wasn’t a concern. Technologically, the benefits are apparent: the grid have some flexibility over when they provide power for storage heaters, and this can be used for load balancing. The consumer is rewarded by a slightly lower rate for heat units vs night time electricity. For example, my current tariff is:

RateApprox usage/yrApprox cost/yr
Day24.223p1276kWh£309.09
Night11.535p1875kWh£216.28
Heat10.469p2169kWh£227.07
Standing charge22.57p365 days£82.38
Total5320kWh£834.82

The “approx usage” I’ve given is based on my usage over the last 12 months. I’m not sure if this is high or low. Scottish power tell me I’m a high user, and MSE would have estimated my usage at about 1/2 of what it is. But also I live alone in a 1-bed flat which visitors complain is cold in winter, and I think I’m quite frugal with electricity – running washing overnight when I can and avoiding heating water on daytime electricity. So who knows. Anyway, this table illustrates the overarching problem with these meters: the tariffs available are uncompetitive. When I’ve tried to switch before, scottish power have told me that there are no alternative tariffs available from them. Other suppliers have said they’re unable to take on my supply, or are only able to offer me a single rate: to which I said no thanks, because running storage heaters on single rate electricity would be stupid, surely? To see just how bad a deal I’m getting, let’s imagine I had an economy 7 meter and all of my heating usage was on night rate instead, as is typical for storage heater installations. This is Vari-Fair by Bulb — the best deal I found from MSE’s comparison:

Unit rateEstimated usage/yrEstimated cost/yr
Day16.46p1276kWh£210.03
Night8.705p4044kWh£352.03
Standing20.558p/day365 days£75.04
Total5320kWh£637.10

So I’m paying around £200 extra per year because of this meter, compared to if I had economy 7. In light of this, I decided it would be worth trying to get the meter replaced, even if I had to incur some of the cost myself. I contacted Scottish Power and asked them whether I could change my meter (with my landlord’s permission) to an economy 7 one. I was repeatedly told that they’d be happy to change my meter, but only if I removed my storage heaters – they insisted that storage heaters could not be used with economy 7. Clearly this is an outright lie – the very point of economy 7 tariffs is to work with storage heaters. I asked them what other options I had for heating my home. I expected them to insist that I needed to use panel heaters, but instead they made the baffling claim that no form of electric heating could be used with an economy 7 meter, and so I’d need to remove electric heaters entirely. Eventually, when I agreed that they could disconnect my storage heaters (with the intention of getting an electrician to reconnect them with a timer afterwards), they told me they’d phone me within a few weeks to arrange replacing my meter. This was over a year ago and I’ve never heard back.

I was pretty angry about this, so I did some digging to find out more about these meters. I found a couple of reports commissioned by Ofgem: The state of the market for customers with dynamically teleswitched meters and Understanding the consumer experience of Dynamically Teleswitched (DTS) meters and tarrifs. These reports come across as quite damning of DTS meters, significantly reflecting my experience. In particular they find:

  • DTS customers often don’t understand how their meter works
  • Customers often didn’t choose to have their meter installed
  • Customers who had tried to switch tariffs had often given up in frustration
  • Tariffs are uncompetitive

As well as this, they found that DTS users are typically elderly, less affluent and less educated. This is worrying from a fuel poverty perspective – in that those who are more likely to struggle with fuel costs are also more likely to be taken advantage of through these tariffs. I’d be willing to bet that a lot of them are renting too, so need for landlord’s permission is another barrier to escaping these meters. The latter report is dated 2014, and I thought it seemed strange that six years later I was still suffering the same frustrating experience, so I contacted Ofgem to ask what they’ve been doing for DTS customers since then. They pointed me to a piece of legislation: Energy Market Investigation (Restricted Meters) Order 2016. This requires that electricity suppliers (with the exception of small suppliers serving fewer than 50k households) offer single-rate tariffs to customers with restricted meters, at no extra cost. This is far from perfect, as a single-rate supply is unlikely to be cost effective for running storage heaters and water heaters, but it’s a start. I was curious, so looked in to just how bad it would be to run my home on single rate electricity. Searching for single rate tariffs with an annual usage of 5320kWh, I found that the best deal was available was Symbio Energy’s “Low, Fair and Green 2020 Variable v7”:

Unit rateEstimated usage/yrEstimated cost/yr
Single rate11.055p5320kWh£588.13
Standing charge24.5p/day365 days£89.43
Total£677.55

Amazingly, it seems like it would still be cheaper to accept a single rate rather than stick with Scottish Power! And that’s even with about 75% of my electricity usage at night, which is well above the national average. If my daytime electricity usage was higher, my current tariff would be even less competitive. However, Symbio are a small supplier and might not be required to take on my restricted meter (if they have fewer than 50k customers). Even still, there are plenty of single-rate tariffs available from big-name suppliers which would cost <£800 a year, so still a small saving compared with my current situation. And like all of the other tariffs I’ve compared here, some of them have no exit charges.

I’m now trying to switch to a single rate tariff. I’m not expecting it to be easy but I’ll see how it goes. When I’ve tried to switch before I’ve run into problems with dual MPANs — so this time I’ve made extra sure that I’ve made the new supplier aware of that. I hope that the situation improves for consumers with DTS meters. Especially with smart meters being rolled out, this could be a good opportunity to allow such customers more flexibility. While I’ m not particularly familiar with SMETS2 meters, my understanding is that they have multiple switched outputs and so could be programmed remotely to switch between DTS, single rate or economy 7 tariffs. I hope that Ofgem will consider taking advantage of this and pushing further legislation to allow us this flexibility and protect us from exploitation. Because what is happening now feels like exploitation of a captive market by Scottish Power, and a cynical attempt too hold on to these customers. The “state of the market” report by Ofgem sugests that 551k homes have DTS meters UK-wide, including 91k in the south of Scotland. We’re constantly reminded of the need to compare energy suppliers in adverts by price comparison sites. But half a million homes don’t have this option. Hopefully there are brighter (and warmer!) days ahead.

Categories
Uncategorized

Kenwood Chef A701a Electronic Speed Controller Retrofit

Following on from my previous post, I gained another addition to my mixer family: a kenwood chef A701a.

Latest addition to my collection on the right

I got this as spares/repairs and when I opened it up, one thing was obviously wrong: it seemed to have swallowed half a bag of flour:

It was everywhere

Once I got it cleaned up, it turned out the fault was just a wire loose. The solder joint had snapped, I suppose. With this soldered back on, it lived again!

Broken wire – the neutral wire from the motor

Running it at a low speed, I could smell the residual flour burning off the 450 ohm wirewound resistor. This got pretty hot in low speed operation because at lower speeds it alternates between having this resistor in series with the motor, and shorting it out. From my research previously, this resistor seems to be the most common failure in these mixers. Aside from that, it’s wasting a tonne of power and making the motor hotter than it needs to be, so it worried me a little from a motor reliability perspective too. I realise it had worked fine for 40+ years and the chances are it would continue to be fine, but I thought it would be a fun project to try and retrofit the A901 TRIAC motor controller to this model.

The other reason I wanted to make this modification was that the design required the physical switch on the motor controller board to switch the full motor current. There’s some suppression circuitry which I suspect is partly to reduce the stresses on that switch, but it still gave me the fear – you could even see it sparking as the motor ran! The TRIAC controlled version, on the other hand, only carries the TRIAC gate current through the switch so the stress on the switch is more or less zero.

This snippet from the service manual shows the wiring of the mixer:

The input chokes don’t exist in my model. Also nor do any of the supression capacitors, except the 0.1uF across the resistor (value not labelled but it’s 450 ohm)

I can only assume the chopper circuit is placed in between the field and rotor windings to take advantage of the interference filtering offered by the windings of the motor – since it’s series wound it makes no odds electrically where it’s placed. The same is done on the A901. I haven’t found a service manual for this but I’ve reverse engineered the following schematic:

The switch in series with the gate is the one controlled by the centrifugal governor. It gets pushed open when the motor hits its target speed.

Since the A701a and the A901 are very similar and the motors seem to be more or less the same, I figured I’d just reproduce the A901 circuit verbatim and see if it works. I found this old solder terminal board on it lying around, and I thought it seemed ideal for this circuit. I already had the components as spares for my A901s. The TRIAC in the original circuit is unknown and has some non-standard marking, so I took a punt on a BT137-800E. In this simple circuit I suspect all triacs with a sufficient voltage and current rating are much of a muchness, and the 800V, 8A rating of this seems like it should be plenty.

I disconnected the motor controller circuit on the mixer and placed this new circuit in instead. It works! Here I’m just using a crocodile lead to simulate the switch actuated by the governor so there’s no actual speed control.

I captured this interesting video too, where you can see how the governor works:

With the basic circuit working, I set about figuring out how to fit it. First, I ripped out the old motor controller components, just leaving the board with the switch:

Next, the TRIAC needed mounting. It didn’t seem to get noticeably hot at all, but since the mixer has plenty of space inside it, I thought I may as well give it some heat sinking just to be on the safe side. I used one of the holes in the board to mount it onto a random bit of aluminium angle I found lying around, with a thermal pad to act as a gap filler:

I found this random bit of metal which conveniently fitted in the pillar which supported the capacitor, and retained an M3 screw:

With the circuit bolted down, annoyingly the TRIAC leads didn’t reach to any of the solder terminals:

This may have been a blessing in disguise though since I’m not sure I’d have been comfortable with the solder joints being exposed to the potential stresses caused by vibration in the mount. Anyway, I wired up the TRIAC with short lengths of wire instead. I’ve also wired in the switch.

And now putting it back together and soldering the various motor/mains input/switch wires:

And it works! Another happy mixer! It gets a lot less hot at low speeds now

I’m pretty happy with this result! If I end up getting another one, I’m tempted to do a bit more of a polished job of it by designing a PCB to go on the old board, and possibly make it easier to upgrade more of them. It seems like A701s appear with broken motor controllers from time to time, and the wirewound resistor is expensive to find a replacement for. Replacing just a PCB could be a good alternative…

Categories
Uncategorized

Kenwood Chef A901 repair

I was in the market for a kitchen mixer a while ago, so when I saw a kenwood chef A901 in a charity shop, I jumped on the opportunity to own a piece of vintage kit like this.

As I walked down the road carrying it home, a woman stopped me to say that she recognises the mixer because it’s the first mixer she bought when she moved in with her husband in the ’60s! I’m not sure it’s that old so she might have been thinking of a slightly different model (maybe the A701) but it still goes to show how well the design has stood the test of time.

It served me well for a little while, before it suddenly started smoking and sounding weird while I was using it. Without the motivation to debug it at the time, I put it back in my cupboard as a project for another day. I feared it would be a failure in the motor insulation so the mixer might be a write-off.

Anyway, I revisited it recently and found that I was a fool to put it off for so long! The fault was a very common one and I fixed mine with little difficulty. Having put the effort into figuring out this fault, I thought maybe it would be fun to fix up some more. So I grabbed another one listed as “spares and repairs” off ebay, and sure enough, it had the same fault. Here’s a few highlights from my fixing it:

First, the top cover needs to come off. I found on my way in that a few parts were quite dirty/greasy. A little bit of soapy water cleaned these up nicely though.

Once the top comes off you can see the motor and the gearbox.

Next we need to get into the other end of the motor housing to slide the motor out. This turned out to be very frustrating. One of the screws was seized pretty badly. My screwdriver kept camming out and the screw wouldn’t turn. Even with penetrating oil, I couldn’t get it out. I was preparing myself for the possibility that I might have to drill it out but out of desparation, I tried a long shot: heating it with a hot air gun. It turned out this loosened it just enough to get it open!

With the motor out of the mixer, you can see that the problem is exactly what you’d expect: a failed capacitor.

In fact, I even found the other half of the capacitor rattling around in the housing!

The failed capacitor was a 150nF X-rated but there was also a 47nF capacitor which I figured I ought to replace too. I had some 47nF and 100nF X-rated capacitors in my parts box so I paralleled up both to replace the 150nF.

While I had the motor out, I figured I’d also check the brushes since these are the first thing to go on the motor. With about 14mm left on each side, these still seem perfectly adequate and don’t need replacement.

Almost ready to put it back together! Before I put the motor assembly back in the housing though, I decided I didn’t fancy putting that screw back in which I nearly wore the head off of trying to get it out: it might never come out again if the mixer needs maintenance in the future! I had a look around the workshop for any suitable replacement screws but I couldn’t find any with the same thread. It looked like some kind of self-tapper. The original screw was slightly smaller than M4 though, so I tapped out the holes to M4 and fitted it back together with M4 bolts instead.

Here’s the motor assembly back in the case with the new capacitors:

I tested it and it worked! But at the lowest speed it was a bit too slow, and it pulsed a lot. The reason is because of the ingenious (if, by modern standards, quite primitive) motor speed control. The motor axle has a kind of leaf spring arrangement on it, whose edges get flung out by centrifugal force (in their reference frame, before any pedants correct me) as the motor spins. This causes the top of the spring to push up on a switch on the control board, and stop the triac firing. So it spins up to a certain speed, then starts skipping mains cycles. It’s essentially an electrical version of a Watt governor or a hit-and-miss engine. Amazing! The effect is very pronounced at low speeds and with no load, since the motor’s inertia makes the control loop quite underdamped but adding a load makes it less noticeable. The speed is set just by adjusting the position of the control board relative to the rotor. To increase the minimum speed a little, I just loosened the spring loaded screws a little until the minimum speed seemed about right, without excessive pulsing.

Here’s the hinge mechanism put back together with nice socket head M4 machine screws, rather than the nasty phillips head self-tappers it had originally:

With it all put back together, there was one final step. The rubber feet on the bottom were completely worn down. It turns out you can buy replacements online, and the gap between the bottom of the mixer and the work surface may contribute to motor cooling. A corkscrew comes in handy for pulling out the old feet:

And it’s all happy again! That was quite a satisfying saturday project, now to find someone in the market for one of these… And maybe get some more and do the whole thing again!

Categories
Uncategorized

Fake flash

Being a complete fanny, I managed to drop my phone and break the display assembly again.

photo5942927736622393988

I was in the pub the next day with a friend recounting my tale of woe, and as he was showing off his shiny new phone to me, it suddenly occurred to me that we used to have the same phone. I ended up buying his old phone off him for parts, and repaired my phone there and then.

photo5945285849761556663
photo5945285849761556664
photo5945285849761556665

Annoyingly, in doing so I managed to snap my SD card!
dsgdfxh

I bought a new 64GB SD card on ebay and when it arrived I decided to do a check of its integrity:

$ dd if=/dev/urandom bs=1M count=64000|pv|sudo tee /dev/mmcblk0|sha1sum -

Already things were seeming suspicious: It claimed to be a class 10 card (10MB/s) but it initially wrote (at least for the first few GB when I was paying attention) at about 2-2.5MB/s.
When I came back later, it was ~45GB in and had sped up to about 15MB/s.

I had planned to read it back and check that the data were the same:

$ sudo pv /dev/mmcblk0 | sha1sum -

But it read at about 10MB/s (rather than the advertised 50MB/s) so I lost interest and tried a different approach, writing to random locations and reading back. 54MiB in was fine, but 54722MiB in returned garbage.

I threw together a quick script to check random 1MiB blocks throughout the disk:

#!/bin/bash
dd if=/dev/urandom bs=1M count=1 of=somerandom
for i in $(seq 1 1000) 
do
	thisblock=$(shuf -i 0-63999 -n 1)
	sudo dd if=somerandom of=/dev/mmcblk0 bs=1M seek=$thisblock
	sudo sync
	sudo dd if=/dev/mmcblk0 skip=$thisblock bs=1M count=1 | diff - somerandom
	if [ $? == 1 ]
	then
		echo "bad"
		echo $thisblock >> badblocks
	else
		echo "good"
		echo $thisblock >> goodblocks
	fi
done

(I don’t know if the sync is necessary – I wasn’t sure if maybe there’d be some transparent caching in play whereby the OS would assume what it’s just written is still there.)

This gave me two lists, one of the known good blocks and one of the known bad blocks. About 3/4 of the blocks were bad, so it was looking like this 64GB was a 16GB in disguise:

 $ wc -l goodblocks badblocks 
 240 goodblocks
 760 badblocks
1000 total

I threw together a piece of octave too, to make a visualisation:

goodblocks=dlmread("goodblocks");
badblocks=dlmread("badblocks");
image=zeros(320,200);
for i=goodblocks'
	x=floor(i/200)+1;
	y=(mod(i,200))+1;
	image(x,y)=1;
end
for i=badblocks'
	x=floor(i/200)+1;
	y=(mod(i,200))+1;
	image(x,y)=2;
end
colourmap=[0 0 0; 0 1 0; 1 0 0];
imshow(image+1,colourmap);
imwrite(image+1,colourmap,"image.png")

The result:
image
Each pixel represents a 1MiB block of the SD card, top left being the start, working right then down, with the bottom right being 64GB (more accurately 64kMiB, or 64000MiB). Black pixels are unknown (the script took a random sample of 1000 blocks out of 64k), green are good (read back the same) and red are bad. This looks pretty conclusively like it’s a 16GB card in disguise.

The bad section seems to be the same 512 byte pattern repeated over and over for a bit, then the repeated pattern changes sometimes for no discernible reason. I haven’t invested much energy into figuring out why yet.

Categories
Uncategorized

Puzzlehunt 2015: Dossier


Puzzlehunt

Since 2012, CUCaTS have been running a puzzlehunt annually during Cambridge’s May Week. Usually lasting 24 hours, teams of up to three compete to solve as many puzzles as possible in the allotted time. The puzzles tend to cover a wide variety of fields, including steganography, number theory, encryption, signal processing, geometry and even linguistics. Some of them are pen-and-paper problems, some require URL hacking and the like, while some require physically going to a place to look for clues. I’ve taken part twice and helped organise it three times. And either side of the admin interface, it’s always been an amazing thing to be involved in.

The puzzle

This post is a writeup for one of the puzzles I set in 2015. This one’s called dossier and you can find it on the puzzlehunt 2015 page.

Uncharacteristically, the puzzle description isn’t that cryptic: you’re told that you need to decipher the two in-depth ciphertexts.

In other words, there are two plaintexts, let’s call them p_0 and p_1. These are both encrypted with the same key, say k for example. This means we have two ciphertexts c_0=p_0 \oplus k and c_1=p_1 \oplus k

Solution

Bitwise XORing the two ciphertexts together gives us the two plaintexts XORed together:

c_0 \oplus c_1 = ( p_0 \oplus k ) \oplus ( p_1 \oplus k ) = p_0 \oplus p_1

So if we know part of one of the plaintexts, we can work out the corresponding part of the key and, helpfully, the corresponding part of the other plaintext.

The body of the puzzle gives a few “crossword-style” clues, along with corresponding locations in the texts at which these words occur. Take the first clue, “0x014 Maniac Hat Time (13)” – this is an anagram of “mathematician”, which occurs starting at the 14th byte of the plaintext.

At this point, enter a little program I wrote to help with solving this: xorsolver. This is just a convenience thing really, and partly just a little exercise in tkinter for me. It does nothing which can’t be done with a few lines of c and a text editor. It’s also a terrible piece of code, with copy and pasted code lines all over the place.

Essentially it allows you to make guesses at parts of the plaintext and immediately see the result the other plaintext.

To start with, it initialises with a guess of the first plaintext as a long run of null bytes. The other plaintext is correspondingly nonsensical.

Screenshot_2017-02-12_18-01-53

As soon as we enter our first piece of guessed plaintext (specifying a memory location and the text at the bottom), a piece of the other plaintext reveals itself:

Screenshot_2017-02-12_18-04-03

” am writing t”. In fact we could guess that the word “mathematician” is surrounded by spaces: guessing ” mathematician ” at 0x13 yields: “I am writing th”. Now we can make a guess at the context for this. Let’s try ” I am writing this” at position 0x12.

Screenshot_2017-02-12_18-18-25

” a mathematician fr”. Guessing a bit further ahead: ” a mathematician from ” gives ” I am writing this le”. Another bit of leapfrogging ahead, and we end up with “, I am writing this letter ” and ” a mathematician from Meche”. Guessing placenames beginning with “Meche” or a word to follow “letter” might be a bit tedious, so let’s leave this section for now and pick up another clue.

Screenshot_2017-02-12_18-25-02

The clue “0x071 Not just a guess (6)” corresponds to “theory”. Guessing this at position 0x71 gives ” week.” in the other text. A decent guess at what comes before this might be ” last”. So we guess ” last week.” at 0x6C. This gives “ring theory”. A bit more leapfrogging and the furthest we can easily get with this section is ” in the fields of string theory and knot theory. He ” and “er I bought from you last week. Unfortunately, the a”.

Picking up some more clues, and doing a bit more leapfrogging, we might end up here with our two texts:

����������������� a mathematician from Meche����������������������������������������#548 in the fields of string theory and knot theory. He %,sm$6/��������������������������������������������������������������������������������������������������������������eI v probably ))-m)j9'?8l.3v��������������������������������������������������������������������tthe first reticulated pantograph ��������������������, this day to be one of the most important inventions of the 19th century in ?? ? ?  ?  theory�����������

and:

��Is$RN>A�, I am writing this letter 
NOO/��MOB'TPE�
��N�V????er I bought from you last week. Unfortunately, the a       D�EdA
�?TELFMI�
FT[@[VONHSI
IALGI
A
TCUKDtSL
	GN>O I&? however, ??????????????�HRFRM
HXVQI
NA"OD=
OHEs
�O
OJ� �' is interfering greatly with my pALNI
YP	DR ial purification and I would be grateful if you could rectify this at once. >+|#$  ",oS*v~vxE#A\�

So it looks like the two texts are a biography of some mathematician/inventor, and a complaint letter. Maybe not the juicy enemy communications we were after but let’s stick with it anyway. Incidentally, there’s no way to know which piece of plaintext corresponds to which text. You could end up with them randomly jumbled up together and have to untangle them – I’ve cheated a bit here since I know both of the texts anyway – although I did still make one mistake along the way and have to correct it. The only way to deal with this is through the context in the surrounding text. If the texts were formed of a load of independent characters, de-interlacing them would prove futile. But generally the structure of words and sentences, and the general style of the piece helps with decoding.

Anyway, using this higher level context, we might guess that the complaint letter would begin with a greeting. With a bit of guesswork we find that “Dear Sir or Madam,” fits perfectly. We end up with a good chunk of plaintext of the mathematician’s biography. “Henri Martens was a mathematician from Meche”.

The rest of the solution proceeds in the same way, using the clues given and guessing pieces of the text. Eventually you end up with the two plaintexts:
“Henri Martens was a mathematician from Mechelen, Belgium. He pioneered numerous advances in the fields of string theory and knot theory. He was awarded the Donald Knuth prize for computing in 1842 for the invention of the impersonal computer. This amazing feat, however, was surpassed shortly after in 1843 when Lord Timpson of Skegness demonstrated the the first reticulated pantograph — widely regarded to this day to be one of the most important inventions of the 19th century in the field of applied maths.”

and:
“Dear Sir or Madam, I am writing this letter to complain about the four dimensional printer I bought from you last week. Unfortunately, the arm meant to move it in the temporal dimension is defective and is stuck travelling forward at the speed of light. You are probably aware that this renders my machine into an ordinary three dimensional printer. This is interfering greatly with my plans for peaceful racial purification and I would be grateful if you could rectify this at once. Sincerely, Abratma Gandhler”

The theme of the latter was intended to change to something a little more lighthearted, along the lines of “…my plans for the enslavement of all of lifekind on earth…” but the wrong files got uploaded in the end.

Writing the puzzle

If I recall correctly, this puzzle was written at around 5AM during the 2014 puzzlehunt. Once I had the idea, two of us separately sat down to write 512 bytes of nonsense. Afterwards, I checked to see how they aligned, in terms of difficult sections matching up. The “leapfrogging” technique can reach a frustrating stall if both ciphertexts reach a difficult-to-guess bit at the same time. It’s much harder to figure that “shortly after in” is followed by “1843 when Lord Timpson of Skegness”, but if you see “I’m writing this letter”, then “to complain about the” isn’t a particularly outlandish guess. So by adding/removing filler in one text, I was able to adjust the difficulty of the puzzle. I also decided to give plenty of clues to give multiple angles of a attack for difficult sections.

Here I’ve coloured the two texts roughly by predictable filler text (green) and stupid high-entropy stuff (red). The clues are also marked in cyan.
plaintexts

You can see that the three sections where red occurs on both lines simultaneously have clues relatively nearby.

In the end, nobody reached this puzzle in 2014 and it was rolled over to the 2015 puzzlehunt. It proved quite popular, with several teams eventually submitting correct answers. If this seems like your kind of thing, sign up to the CUCaTS mailing list to get more information closer to the time.

Categories
Uncategorized

Bullshit on the Internet

You see a lot of bullshit on the internet. Some of it genuine scams, some of it well-intentioned but ultimately misguided.

Well, here’s something I’m not quite sure what to make of: the Eco-Cooler.

The video demonstrates a design of “air conditioning” requiring no electricity: instead just relying on the flow of air through bottles. It’s one of those things where at first glance (at least if you watch their explanation) it looks plausible, then you think about it a bit more and go “wait, what?”

Their snazzy video explains… blow on your hand with your lips pursed and it feels cooler than if you have your mouth wide open.
ecocooler-screenshot

That, they say, is how their cooler works.

ecocooler-screenshot2

Air blows through a bottle, it gets compressed, and that cools it down. A nice idea, but unfortunately thermodynamics and fluid mechanics don’t work like that.

Let’s look at the first example – air blown through pursed lips. Why does it feel colder? The answer is simply that it’s faster. Faster fluids transfer more heat, because they keep replacing the warm air around you. On a cold day, you warm up a small “bubble” of air around you. When the wind blows, it blows that away and suddenly you’re surrounded by cold air again. The faster that wind, the less of a “bubble” of warm air you can maintain and the colder you feel. But the air you’re blowing on your hand isn’t any colder because you’ve blown it through pursed lips.

Refrigeration Basics

So what about “the bottle neck compresses and cools the air”? This seems like a massive misunderstanding of how a refrigeration cycle works. So let’s start there.

The way conventional air-conditioning works is that it takes some working fluid (1) and compresses it (2), using a pump. The pressure increases and, crucially, the volume decreases. The same amount of heat* in a smaller space is going to get hotter. If you dump this heat somewhere (3) and bring the compressed fluid down to ambient temperature, it now has less heat than it started with. This is what’s happening in the compressor outdoors when you have an air con system and why it blows out hot air. Since the now cool-ish fluid is still compressed, you can expand it again. When you do, that same amount of heat* spreads out across a larger volume, and so you end up colder than you started. This is what’s happening indoors where the cold air comes out.
(*Actually the heat changes a bit because of the work done on or by the fluid, but that’s unnecessary complication and isn’t crucial to understand the general idea. I thought I should say it anyway for the sake of intellectual honesty.)
all-numbered

So is that what’s happening here? No. The key thing to note about a refrigeration (or air-conditioning) system is that no heat is destroyed. The inside’s made colder at the expense of hot air being blown out of the compressor. So, if this eco-cooler does indeed cool air, where does the heat go?

Could we potentially make this work if we extracted some heat at the bottle neck, and dissipated that to the outside environment?

Flow Analysis

Enter everyone’s favourite often-misused equation of fluid mechanics: Bernoulli.

Bernoulli’s principle is simply a statement that along a streamline, energy is conserved. This means that anywhere along the flow, \frac{v^2}{2}+\frac{p}{\rho} is the same. As air travels through one of the bottles, it will indeed have to speed up as it enters the mouth, because of conservation of mass and therefore volume (subsonic air flow is essentially incompressible). By Bernoulli, this causes pressure to decrease and therefore, according to the ideal gas law, the temperature to change slightly.

How much does the temperature change? Let’s say air enters at 5m/s. That’s a fair breeze. Looking at a drinks bottle, i’d guess the mouth is about 1/5 the diameter of the bottle, so the air has to accelerate to 25 times its starting speed to get through. Let’s also assume an air density of 1.2kg/m³ and ignore the obvious ludicrousness of air travelling through a bottle neck at 125m/s (in reality viscosity would mean a lot more energy gets turned to heat).
Plugging the numbers in, the gauge pressure in the mouth is:
\rho \left(\frac{v_1^2}{2}-\frac{v_2^2}{2} \right) = 1.2 \left(\frac{5^2}{2}-\frac{125^2}{2} \right) = -9.4kPa

From the ideal gas equation, air at 45 degrees will be around 110kPa:
P=\rho \overline{R}T=1.2 \times 287\times(45+273)=109.5kPa

Taking off the gauge pressure and doing the same in reverse gives a temperature of 18 degrees. The air in the mouth is actually colder! This isn’t what we want for a refrigeration cycle at all — if anything this colder air would absorb heat from the environment and be even hotter than it started when it leaves the mouth and returns to atmospheric pressure. We’ve made a very shitty heat pump.

What is this thing?

So if this isn’t actually going to do any refrigeration, what is it? And why does it cool the building by 5 degrees as they claim in the video?

My guess is that their test is unfair or unrepresentative. Perhaps they were comparing no ventilation with this, rather than comparing this with… well, just a window. Just the circulation of cooler air from outside is going to help to some extent, but no more than an open window. But they don’t seem to have any information beyond what’s in the video.

I struggle to work out what the motivation is here. Initially it seems like a solar-freakin-roadways sort of thing: well-intentioned but fatally flawed. But although there’s nothing being sold here, the video does mention two sponsors, so perhaps this is just a publicity stunt for them?

They seem to have a website which is under construction at the moment, so perhaps we’ll find out soon…

Categories
Uncategorized

Nixie Tube Clock

Today, the story of Ahmed Mohammed has been doing the rounds. I suppose you could even say it’s gone “viral”, as I believe is the modern parlance. A tl;dr for those who missed it: 14-year old kid and electronics enthusiast takes a homemade digital clock into school to show to his teachers. Far from being impressed at his ingenuity and enthusiasm for technology, they decide it looks too much like a bomb and so the police are called. All of which begs the question, would this have happened if he was white? A certain clickbait news website picked up on this idea with a headline “7 Kids Not Named Mohamed Who Brought Homemade Clocks to School And Didn’t Get Arrested”, upon reading which I realised — hey, that’s me! So I thought this seemed as good a time as any to dig out the old photos of the nixie tube clock I made back in 2010 — something I’ve been meaning to do for a while.

And yes, I took this into school to show my electronics teacher. And no, I wasn’t arrested!

Image0499
This is a nixie tube. It’s a neon gas filled tube containing an anode grid and several symbol (in this case number) shaped cathodes. When a negative voltage is applied to one of the cathodes, the neon in its proximity is excited and glows. It’s a very pretty type of display since the symbols are bent out of metal rather than formed of straight line segments, so the numbers have a very natural and appealing shape. The tubes I used were soviet surplus ИН-16 tubes.

Image0504
For the physical design of the clock, I wanted to go for a slightly steampunk look, using copper pipes. I soldered this up out of 15mm plumbing pipe and joints, and the idea was that I’d thread all the wires through the pipes so the tubes were sticking out the top. Ultimately I planned to make a wooden box — stained mahogany — for the electronics with this sticking out of the top.
Trying to fit all of those wires through 15mm pipe was a real pain!

Image0518
To drive them, I commoned up all of the anodes and put a high voltage NPN transistor on each cathode.
In hindsight, I could have grouped the cathodes and had a high side switch to each anode, then scanned through them, much like one would do with a multi-digit seven segment display (check out the wiring of these 4-digit displays for example) but I didn’t think of that at the time and probably wouldn’t have been able to manage the gate drive anyway. (Give me a break, I was still a high school kid at the time!)

Image0523
Here I am testing the drive circuit. I know that isn’t a real time, I just picked random digits! It was powered here (and indeed onwards) from a crude mains isolation setup: I stepped the mains down with a transformer to low voltage DC, and took two taps off that: one went through an identical transformer in reverse to step it back up to isolated 230VAC (then rectified and smoothed) and the other went to a rectifier/regulator circuit to provide 5VDC for the digital electronics.

Image0565
At this point the photo album was inexplicably interrupted by a photo of one of the mice I had at the time. She gave birth to 13 pups shortly after.

Image0554
Here’s the circuit with some of the digital electronics. A PIC microcontroller was the brains of it. (The colourful twisted wires are for in-circuit programming.) It drove the 10’s of hours and 10’s of minutes directly, whereas the digits, to save on microcontroller outputs, were connected up to 4017B decade counters. These were clocked and reset by the PIC, requiring only two outputs each.

Image0570
It took pride of place on our mantelpiece for 24 hours to test for mistakes in my code, and that it kept time reasonably well. The speaker was added to make it “tick”. You can also see the crude isolation transformer setup on the left.

My plan next was to make a PCB for the circuit and make a box for it to go in. But after a brief trip into school to show it off to my teacher, I took it into London Hackspace to work on it — and perhaps solicit some advice on the woodwork side of it — but shortly after that it disappeared! Perhaps it got deleted in a fit of rage tidying, who knows…

Looking back, it’s kind of embarrassing to see how past me did things. Notably the mass-of-wires breadboarding and the bombsite of a workbench while working with mains voltage. There are plenty of things I’d do differently too – probably an SMPS to drive the nixies to save on transformers, commoning up the cathodes and scanning the display, using an RTC (this just used the PIC’s built in clock so kept time really badly!) and in hindsight that copper pipe should /really/ have been grounded, with the single insulated cable running through it at 300VDC. But hey, you live and learn. Five years on, I’m older and wiser. Nothing now remains of this project but the photos and an unused bottle of mahogany wood stain. The hardware all went missing, the code is long gone and the schematics only live on as vague recollections in my head. And it will probably remain so — I have little motivation to make a clock now (where would I put it? why would I need one practically everything shows the time these days?) and plenty of others have done nixie clocks such that they’re almost commonplace nowadays, so the novelty of it has somewhat worn off…

Categories
Uncategorized

pwgen-h

xkcd password strength comic

Inspired by xkcd #936 (reproduced above, credit to xkcd), I threw together a quick python script to generate correct horse battery staple style passwords.

pwgen-h:

#!/usr/bin/python3
import os,random,sys
dictfile=open('wordlist')
dictlist=[]
numword=0
for a_line in dictfile:
	dictlist.append(a_line)
	numword+=1
for x in range(int(sys.argv[1])):
	random.seed(os.urandom(512)) #reseed the RNG each time to ensure cryptographically secure random numbers
	print(dictlist[random.randrange(numword)].strip(),end=" ")
print("")

The name comes from the unix pwgen utility, but -h because it generates “human readable” passwords.

It needs to be fed a file “wordlist” containing one word per line for it to choose from. You can use mine (roughly the 5000 most common english words) or your own. It’s then invoked as pwgen-h num_of_words

Example invocations and outputs:

$ ./pwgen-h 6
starting give progress limit accommodate code
./pwgen-h 4
gravity Latin convenience exclude

The wordlist I used is about 5000 words, so a resulting 6-word password has an entropy of 6 \times log_2{5000}=73.7 bits. That means that there are around 10^{22} possibilities to brute force. At a million guesses per second, it would take in the order of a billion years to check every possibility.

This has the caveat though that you have to take the first password it offers you* for the working above to be valid. If you keep trying it until you see a password which you like, you’re reducing your entropy in a difficult to quantify way.

*Or rather, don’t decide based on the output whether to use it. Of course, you can play around with it as much as you like, but you should decide which password you’re going to use before it’s generated for maximum security.

Categories
Uncategorized

Measuring Capacity of Constellations

Having just the other day finished off my fourth year project (and in doing so, completed my degree! hooray!), I thought I’d share a little tidbit from it which perhaps might be useful to someone else.

Background

Part of the project involved QAM constellation shaping. That is, moving points around in a constellation — in this case, to increase the capacity.
A typical QAM constellation would look something like this, shamelessly taken from wikipedia (created by user Splash):
qam16

Whereas here is an example of a shaped constellation which I investigated (coloured by decision region for hard decision decoding):
64qam-nonsquare

Capacity

Comparing these constellations required that I be able to measure their capacities. Here I’m considering a very simple system:
system
It’s simply a two-dimensional AWGN channel.

Grid Evaluation

This is the first method I tried. It didn’t work very well, and I’ll explain why.

The capacity of the channel is definied by the mutual information between the input and the output:
I(Y;X)=H(Y)-H(Y|X)

Here, H(Y|X)=H(X+N|X)=H(N), which is simply the entropy of the gaussian distribution and has an analytic form. In this case, where the gaussian is bivariate and isotropic (that is, the \Sigma matrix is diagonal), this becomes:
1+log(2\pi\sigma^2)

H(Y) on the other hand is a nastier affair – it’s the entropy of the noisy signal, or that of a sum of gaussians, -\iint{f_Y(y)log(f_Y(y))dy}. This has no closed form so instead the approach I took was to sample f_Y(y) over a grid, and then sum across it for -\sum{f_Y(y)log(f_Y(y))}

This worked quite nicely for some signals, like this one: pdf-good
Most of the probability mass is included in the sampled grid (in fact it sums to 0.9922 here), so the summation serves as a good approximation to the integral.

But, at low SNR, a lot of probability mass spills over the sides:
pdf-bad
Here the probability mass inside the grid only sums to 0.6353. This gave utterly meaningless results (as I’ll show later). I also tried re-scaling the probabilities so that they’d sum to 1, but ultimately this didn’t help either.

Monte Carlo Method

When I mentioned this to my supervisor, he mentioned another possible method, using Monte Carlo instead.

First, flip the mutual information expression around and consider it the other way:
I(X;Y)=H(X)-H(X|Y)

In this case H(X) is easy — X is drawn equiprobably from an alphabet size of M. So H(X)=log(M).

H(X|Y) is the tricky part here, but… we can find it by Monte Carlo.
Pick a load of random points from the joint f_{XY}(x,y):
montecarlo
(This is an example from one of my optimised constellations)

Then for each point, -log(p_{X|Y}(x|y) is calculated and the average is taken. Since the process is ergodic, this tends to the expectation, -\mathbb{E}log(p_{X|Y}(x|y))=H(X|Y)

Comparison

The Monte Carlo method produced much more sensible results:
allmethods

Where the grid evaluation method gives meaningless negative capacities at low SNR, the Monte Carlo method gives results tending gracefully to zero.

In hindsight, this could probably have also been done using the original capacity expression, but calculating H(Y) by Monte Carlo… I’ll leave that as an exercise for the reader.

Results

Just for a little bit of fun, here’s a few constellations of different orders compared using the method above.
comparison

That’s about it for this post. I’ve run out of enthusiasm to write any more. In fact I did about half an hour ago. Maybe it shows. I almost gave up and deleted the post but I figured I’d finish it in case anyone cared. Do let me know in the comments if you found this useful! It’s good to know what posts interest people, and it helps for motivation too…

Here’s my MATLAB code for the two implementations. It’s not particularly tidy and the call to mvnpdf breaks in octave, but perhaps it will be useful as a reference for someone:
ConstellationInformation.m, ConstellationInformationMC.m