Fix: AccessToKeyVaultDenied when using KeyVault with Function App application setting

After following the instructions on the MS website to establish a KeyVault reference and place that in my App Settings, I set up a Managed Service Identity and grant that identity access to my KeyVault key. Next, wishing to follow Microsoft’s advice and secured a firewall around the KeyVault, ensuring that I checked the Allow trusted Microsoft services to bypass this firewall? setting, however, I was still receiving an AccessToKeyVaultDenied error:

Screen shot showing that System assigned managed identity is receiving the AccessToKeyVaultDenied error with the explanation 'Key Vault reference was not able to be resolved because site was denied access to Key Vault reference's vault.'

I even checked and yes, App Service is supposed to be able to bypass the firewall – so what was going on? Well, on the KeyVault resolver reference page it has this text:

Key Vault references are not presently able to resolve secrets stored in a key vault with network restrictions.

That seemed ok when i first read it as after all, there’s an explicit setting to bypass the firewall. But when i disabled network firewall (allow access from all networks), everything suddenly worked, and the key status is Resolved with a nice green tick:

Screen shot showing that the KeyVault key status is "Resolved"

The 9 Ps

I often have conversations with friends and colleagues about their careers. And many times, i point people to a great blog post by my colleague Liz Aab, about the “7 Ps”. But i always find myself adding two to the list, so i thought i’d just post it here.

There are lots of factors which go in to choosing a job. You can’t have all of them, all of the time. At least, i think you can’t. But you can (and should) decide which are most important to you. Here are Liz’s 7 Ps (which she says were originally 5 Ps from some other source). I’ve added my two on the end, and i’ve reworded some of Liz’s original post:

  1. Place : Where geographically do you want to work? The city/country you are based in and your commute affect how you spend your time, and who you spend your time with, both inside and outside work.
  2. People : Who specifically would you work with on a daily basis? Do you like them? Does your boss care about you and want to see you succeed?
  3. Pay : Does the job or sector pay you enough to live the life you want? If not, will your pay will increase in a few years in this career path? Or, are you happy to change your lifestyle to accommodate a lower salary?
  4. Progression : Will you develop skills, knowledge, a network or a reputation that will help you move forward in your career? Does this job offer defined progression opportunities, or do you need to develop these for yourself? If so, are you comfortable with this?
  5. Perception : How do people react when you tell them what you do? Whose opinion do you really care about, and how important is that to you? Of course perceptions of jobs and industries change over time.
  6. Purpose : What is the company or organisation trying to achieve, and do you support that? It’s not just millennials that want to work on something they believe in.
  7. Procedures : In Liz’s list, this is how you do your job day to day. I’ve reworked it – for me, procedures is how the organisation operates. Do they expect a rigid 9-5, or are you trusted to deliver a result? Do decisions get made once and then implemented, or does it take a consensus to make change? Are you empowered to deliver, or do you need permission to take a bathroom break?
  8. Projects : While procedures might be how the work gets done, this is what you’re actually doing. Are you spending your day on the phone, or sitting reading stacks of paper, or crunching Excel, or standing on your feet in front of 25 teenagers? Is your work indoors or outdoors? And do you like doing those things?
  9. Pace : Is it frantic from the moment you wake to when you sleep? Or is there lots of space in the day for you to collect your thoughts or think things through? Are you expected to check your emails after hours, or do you ‘clock off’ when you’re done? What do you need to thrive?

Of course, as Liz points out, what you value today will differ to what’s important to you tomorrow. When you’re young and eager, you may want a role which is always on the go (high pace), and with a compelling purpose. If you start to plan a family, pay and progression move up the list.

Fix: This must be accepted explicitly before updates for this repository can be applied

Some repos, such a the one for the Unifi Controller, use different ‘field’ values to tie a release and require manual updates. For someone like me who has a standalone, automated controller setup designed mainly to keep the firmware up to date without much intervention, this is a hassle. It looks something like this:

[email protected]:~$ sudo apt-get update
[sudo] password for robert: 
Hit:1 bionic InRelease
Get:2 bionic-updates InRelease [88.7 kB]          
Get:3 bionic-security InRelease [88.7 kB]        
Get:4 bionic-backports InRelease [74.6 kB]                                       
Ign:5 xenial/mongodb-org/3.4 InRelease                                         
Hit:6 xenial/mongodb-org/3.4 Release                                           
Get:7 stable InRelease [3,024 B]        
Reading package lists... Done                             
E: Repository ' stable InRelease' changed its 'Codename' value from 'unifi-5.12' to 'unifi-5.13'
N: This must be accepted explicitly before updates for this repository can be applied. See apt-secure(8) manpage for details.

It’s an easy fix. Just tell apt-get to ignore the codename field:

[email protected]:~$ echo 'Acquire::AllowReleaseInfoChange::Codename "true";' | sudo tee    /etc/apt/apt.conf.d/99releaseinfochange
Acquire::AllowReleaseInfoChange::Codename "true";

and then it works!

[email protected]:~$ sudo apt-get update
Ign:1 xenial/mongodb-org/3.4 InRelease
Hit:2 bionic InRelease                                                           
Hit:3 bionic-updates InRelease                                                   
Hit:4 xenial/mongodb-org/3.4 Release                                           
Hit:5 bionic-backports InRelease                                                 
Hit:6 bionic-security InRelease                                                 
Hit:7 stable InRelease                                                           
Reading package lists... Done

Fix pyodbc.Error: (‘01000’, “[01000] [unixODBC][Driver Manager]Can’t open lib ‘ODBC Driver 13 for SQL Server’ : file not found (0) (SQLDriverConnect)”)

I was connecting from my macbook to a SQL Azure Database when i hit the following error:

>>> environ.get('cloud_sql_conn_string')
'Driver={ODBC Driver 13 for SQL Server};,1433;Database=cloud_scales;Uid=<username>;Pwd=<password;Encrypt=yes;TrustServerCertificate=no;Connection Timeout=30;Authentication=ActiveDirectoryPassword'
>>> import pyodbc
>>> cnxn = pyodbc.connect(environ.get('cloud_sql_conn_string'))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
pyodbc.Error: ('01000', "[01000] [unixODBC][Driver Manager]Can't open lib 'ODBC Driver 13 for SQL Server' : file not found (0) (SQLDriverConnect)")

The solution was to install the ODBC driver, following the instructions on the Microsoft website:

brew tap microsoft/mssql-release
brew update
HOMEBREW_NO_ENV_FILTERING=1 ACCEPT_EULA=Y brew install msodbcsql17 mssql-tools
ACCEPT_EULA=Y brew install [email protected] [email protected]

Fix: unable to kmem_alloc enough memory for scatter/gather list in ZFS Solaris 10.5

The ZFS Pool on my server was showing degraded state. After checking the SMART status of the constituent drives and finding no problem, I discovered that there’s a bug in Solaris 10.5 where the system reports a growing number of errors and eventually fails the pool. dmesg shows an error unable to kmem_alloc enough memory for scatter/gather list, however, there is actually nothing wrong with the pool. Running zpool status shows degraded state:

[email protected]:~# zpool status
  pool: rpool
 state: ONLINE
  scan: none requested

        NAME        STATE     READ WRITE CKSUM      CAP            Product /Disks     IOstat mess          SN/LUN
        rpool       ONLINE       0     0     0
          c1t0d0    ONLINE       0     0     0      32.2 GB        VMware Virtual S   S:5 H:25 T:0         000000000000000

errors: No known data errors

  pool: tank
 state: DEGRADED
status: One or more devices has experienced an unrecoverable error.  An
        attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
        using 'zpool clear' or replace the device with 'zpool replace'.
  scan: scrub repaired 0 in 12h15m with 0 errors on Fri Dec 21 00:08:43 2020

        NAME                       STATE     READ WRITE CKSUM      CAP            Product /Disks     IOstat mess          SN/LUN
        tank                       DEGRADED     0     0     0
          raidz1-0                 DEGRADED     0     0     0
            c0t50014EE20BF0750Dd0  ONLINE       0     0     0      4 TB           WDC WD40EFRX-68W   S:0 H:0 T:0          WDWCC4E6NAXVAS
            c0t50014EE263348A3Ed0  ONLINE       0     0     0      4 TB           WDC WD40EFRX-68W   S:0 H:0 T:0          WDWCC4E0FRRRRP
            c0t50014EE2B69D2D68d0  DEGRADED     0     0    20  too many errors      4 TB           WDC WD40EFRX-68W   S:0 H:0 T:0          WDWCC4E3AN2Y99

errors: No known data errors

Running zpool clear recovers the pool:

[email protected]:~# zpool clear
[email protected]:~# zpool status    
  pool: rpool
 state: ONLINE
  scan: none requested

        NAME        STATE     READ WRITE CKSUM
        rpool       ONLINE       0     0     0
          c1t0d0    ONLINE       0     0     0

errors: No known data errors

  pool: tank
 state: ONLINE
status: One or more devices has experienced an unrecoverable error.  An
        attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
        using 'zpool clear' or replace the device with 'zpool replace'.
  scan: none requested

        NAME                       STATE     READ WRITE CKSUM
        tank                       ONLINE       0     0     0
          raidz1-0                 ONLINE       0     0     0
            c0t50014EE20BF0750Dd0  ONLINE       0     0     2
            c0t50014EE263348A3Ed0  ONLINE       0     0     0
            c0t50014EE2B69D2D68d0  ONLINE       0     0     0

Wiring a Yale Keyless Connected Smart Lock to the mains

For various reasons, not least because I wanted to play with it, we have a Yale Keyless Connected Smart Door Lock with a Z-Wave module (we have the v1 module which works fine). This lock has a couple of key features that we liked:

a hand places a round tag next to a lock which is lighting up
  • You can grant or revoke access using RFID tags or cards, or by entering a 6-8 digit code on the keypad.
  • With the Z-Wave module (and a compatible Z-Wave controller), you can programatically add and remove codes so that you can enable codes at specific time or dates. For us, this meant we could create a code for the cleaner, but if they turned up at 2am on a Saturday, the door wouldn’t open for them.

It’s connected to our Samsung SmartThings hub, and i run the RBoy Apps custom device type and smart app to enable the scheduled key rotation etc. Overall, we’ve been fairly happy with it, but the thing really does eat up batteries, and I started to feel guilty about putting between 4 or 8 AA batteries in the bin each month. Of course I also got annoyed at constantly having to buy them and change them, so I decided to try rechargeables.

We bought some Panasonic Eneloop Pro batteries. I’d read a very interesting piece of research showing how high performance NiMH batteries actually outperform alkaline batteries – delivering a stable ~1.2v for far longer. As it turns out, this is a problem.

With a regular battery, as the charge drops, the device detects this and fires off an alert reminding you to change them. As the research showed, however, NiMH batteries provide a fairly constant 1.2v until the “power” in the batteries is pretty much depleted, and then they just die. This isn’t a problem for a radio controlled car. But of course if the batteries go flat on your front door lock, you can’t get in to your house as there’s no key override on it, and with no alerts, we wouldn’t know to change them. Although you can power the lock from the outside in an emergency using a 9v battery, after a particularly embarrassing situation where I discovered that the 9v battery I had stored in the glove box had expired 2 years ago, I decided to figure out how to wire the lock up to a permanent power supply. The main challenge here of course is that I would like to be able to unlock my house even when the power is out. After a bit of thinking, i decided that I probably needed a battery in there somewhere too.

Picture from ebay seller random-bargains2009

The first challenge is working out how to wire up the device. Ideally, didn’t want to be soldering connectors on. After a bit of research, I found a “4 X 6V AA MONEY SAVING BATTERY REPLACEMENT PLUG IN ADAPTER” on ebay (the item i bought is from “random-bargains2009” but there were three or four different ones from other sellers). This is basically an AA battery with a wire coming out of it connected to a mains adaptor, and 3 “dummy” battery blanks. I cut off the AC adaptor, and soldered on a USB A plug, and connected it to a Belkin USB battery pack, then plugged that in to a charger to keep it constantly topped up and … nothing. It turns out that Belkin battery packs can’t provide power and be charged at the same time. Doh!

I tried another battery pack, and all was fine until my wife tried to come in about 5 minutes later. Apparently the “smart” charge controller built in to the battery didn’t detect sufficient current, and so switched off the battery pack. Brilliant.

After a bit of research, I was able to find a 3,000mAh battery pack with a USB plug (from Amazon) that was designed to provide backup power for 12v CCTV cameras. Made by Chinese company TalentCell, it claims CE compliance for both the batteries and the charger. Mine arrived from Germany with an EU plug, but I already had some fused, screw fixed adaptor plugs, so not a problem.

Finally, the I ran the cable around the frame of the glass in the door to try and keep it discrete, and I’m now confident that we won’t ever be locked out again.

How 5G connectivity and new technology could pave the way for self-driving cars

Hybrid peer-to-peer/5G vehicle communication technology, C-V2CX ("cellular-vehicle-to-everything"), has evolved since it's 2016 debut, with recent demos showing how it helps vehicles "see" threats and obstacles out of sensor range (such as cars coming around corners, traffic lights and so on). But it's not the only protocol on the block – Toyota, the world's largest car manufacturer, and GM, prefer a competing protocol based on wifi. The winner should start to emerge later this year when 5G trials begin.
C-V2X enables vehicles to communicate, which should reduce accidents and aid autonomous driving.

IBM researchers build AI-powered prototype to help small farmers test soil | ZDNet

IBM is testing a paper test strip which, when analysed with an app on a standard smartphone, could reduce the time and cost for farmers trying to work out how to prepare soil for planting, treat water, or maintain optimal growth of crops. The card is about the size of a business card, and changes colour in specific patterns to measure pH, nitrogen dioxide, aluminium and other chemicals necessary for (or best avoided) for healthy plant growth. The app allows immediate, precise diagnosis (the camera is more a accurate colour sensor than the human eye), and aggregated data can help governments monitor fertilizer/chemical use.
The IBM AgroPad is a paper testing strip that, when combined with a mobile app, relies on machine vision to measure the precise amounts of chemicals in samples of water and soil.