Apple’s new robot, Daisy (an upgraded version of a robot announced in 2016), can disassemble 200 iPhones per hour, sorting the parts for reuse and recycling, extracting parts which traditional recyclers are unable to safely or cost effectively remove.
https://rob.al/2HOCTnO
Apple has created a new robot – not for building products, but for ripping iPhones apart. The robot, named Daisy, can take nine different iPhones models apart and extract the important parts of them,…
Blockchain Is About to Revolutionize the Shipping Industry
The move to paperless movement of goods in these extraordinarily complex logistics chains has the potential to unlock massive benefits, reducing the cost of movements, by shortening the time it takes for goods to cross checkpoints and borders – an example 32 day journey of perishable goods could be completed in 20.
https://rob.al/2JZagVB
Globalization has brought the most advanced trading networks the world has seen, with the biggest, fastest vessels, robot-operated ports and vast computer databases tracking cargoes. But it all still…
A Taxonomy of Tech Debt
I have no idea about the game that the author is engineering manager for, but this is a good write up of a framework for assessing technical debt, including the value of resolving it now vs later (never)
https://rob.al/2ELJgox
Bill Clark discusses classifying and managing tech debt at Riot.
Musk Says Excessive Automation Was ‘My Mistake’
While I’m sure that eventually, the process could have been automated, anyone who’s tried this even at a trivial scale would tell you to get the process working first, then to automate. So I don’t get how Tesla ended up in this position.
https://rob.al/2IW0qCf
Tesla Inc.’s Elon Musk, who’s built up an aura around how automated his car assembly plant will be, has good news for humans: We still need your help.
Your Next T-Shirt Will Be Made by a Robot
The clothing industry has long relied on low cost labour in developing countries, but advances in industrial robotics and machine learning and vision, at reducing cost, mean even those jobs are now at risk.
https://rob.al/2JGpjTO
Georgia Tech spin-off SoftWear Automation is developing ultrafast sewing robots that could upend the clothing industry
China’s Uber says it’s time to go global
"For the past two decades, it was China who learned more from the US," he says. "But in the next 10 years, we'll ride on each other's successes. There's no point thinking who will surpass who."
https://rob.al/2EIUv0S
The head of China’s ride-hailing giant Didi Chuxing says his team is young, idealistic and full of surprises.
The NVIDIA GPU Accelerated Cloud (NGC) for Deep Learning and HPC
An interesting move from hardware vendor to PaaS provider: The NVIDIA GPU Accelerated Cloud (NGC) for Deep Learning and HPC
https://rob.al/2tnW3gH
A GPU-accelerated cloud platform with access to catalog of fully integrated and optimized containers for deep learning and HPC frameworks. Learn more.
LetsEncrypt with DNS-01 validation for Unifi
Update 2021-01-08: this is now out of date. See my updated post with a much easier method.
I have a number of Ubiquiti UAPs, and I manage them with the UniFi app, installed on a linode server. Like any publicly hosted server, i want to use a trusted SSL certificate, and for that, I chose LetsEncrypt with DNS-01 validation, as i found a useful helper script by thatsamguy on the UniFi forums. I use AWS Route53 to host the DNS zone.
UniFi doesn’t have built in support for LetsEncrypt, so I put together a simple solution using the DNS-01 validation method. Here’s how i did it:
- Created a new, unprivileged user on the host:
sudo adduser dehydrated - Created the directory to store the certificates. I chose to use the default /etc/letsencrypt folder
sudo mkdir /etc/letsencrypt - I granted the dehydrated user full access to this folder:
sudo chown dehydrated:dehydrated /etc/letsencrypt - installed the dependencies which are in the repo:
sudo apt-get install --no-install-recommends jq sed findutils s-nail - installed the cli53 dependency which is not in the repo by following the instructions on the git readme.md file
- logged in as the dehydrated user:
su - dehydrated - fetched dehydrated and made it executable:
wget https://raw.githubusercontent.com/lukas2511/dehydrated/master/dehydrated chmod +x dehydrated
- fetched the dehydrated route53 hook script and made it executable:
wget https://raw.githubusercontent.com/whereisaaron/dehydrated-route53-hook-script/master/hook.sh chmod +x hook.sh
- Created a new IAM access policy in AWS. I found that the sample policy given with the root 53 hook readme.md didn’t work – here’s the policy that I added:
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "route53:ListHostedZones", "route53:ListHostedZonesByName", "route53:ListResourceRecordSets" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "route53:ListResourceRecordSets", "route53:ChangeResourceRecordSets" ], "Resource": "arn:aws:route53:::hostedzone/*" }, { "Effect": "Allow", "Action": [ "route53:GetChange" ], "Resource": "arn:aws:route53:::change/*" } ] } - Created a new IAM user in AWS and assigned the policy to them (and only this policy), taking a note of the AWS Access Key and AWS Secret Access Key.
- Created a new file to store the key:
mkdir ~/.aws nano ~/.aws/credentials
The file looks like this (obviously, put your own data in there):
[default] aws_access_key_id = AKIAIJFAKGII4MDS3KHA aws_secret_access_key = Q+XoOGa5J3AS39as593Ds1f5F91zRy0btkfW
- Created a new config file:
nano ~/config
The file looks like this (edited from the sample):
######################################################## # This is the main config file for dehydrated # # # # This file is looked for in the following locations: # # $SCRIPTDIR/config (next to this script) # # /usr/local/etc/dehydrated/config # # /etc/dehydrated/config # # ${PWD}/config (in current working-directory) # # # # Default values of this config are in comments # ######################################################## # Resolve names to addresses of IP version only. (curl) # supported values: 4, 6 # default: #IP_VERSION= # Path to certificate authority (default: https://acme-v01.api.letsencrypt.org/directory) #CA="https://acme-v01.api.letsencrypt.org/directory" # CA="https://acme-staging.api.letsencrypt.org/directory" # Path to certificate authority license terms redirect (default: https://acme-v01.api.letsencrypt.org/terms) #CA_TERMS="https://acme-v01.api.letsencrypt.org/terms" # CA_TERMS="https://acme-staging.api.letsencrypt.org/terms" # Path to license agreement (default: ) #LICENSE="" # Which challenge should be used? Currently http-01 and dns-01 are supported CHALLENGETYPE="dns-01" # Path to a directory containing additional config files, allowing to override # the defaults found in the main configuration file. Additional config files # in this directory needs to be named with a '.sh' ending. # default: #CONFIG_D= # Base directory for account key, generated certificates and list of domains (default: $SCRIPTDIR -- uses config directory if undefined) #BASEDIR=$SCRIPTDIR # File containing the list of domains to request certificates for (default: $BASEDIR/domains.txt) #DOMAINS_TXT="${BASEDIR}/domains.txt" # Output directory for generated certificates #CERTDIR="${BASEDIR}/certs" # Directory for account keys and registration information #ACCOUNTDIR="${BASEDIR}/accounts" # Output directory for challenge-tokens to be served by webserver or deployed in HOOK (default: /var/www/dehydrated) #WELLKNOWN="/var/www/dehydrated" # Default keysize for private keys (default: 4096) #KEYSIZE="4096" # Path to openssl config file (default: - tries to figure out system default) #OPENSSL_CNF= # Extra options passed to the curl binary (default: ) #CURL_OPTS= # Program or function called in certain situations # # After generating the challenge-response, or after failed challenge (in this case altname is empty) # Given arguments: clean_challenge|deploy_challenge altname token-filename token-content # # After successfully signing certificate # Given arguments: deploy_cert domain path/to/privkey.pem path/to/cert.pem path/to/fullchain.pem # # BASEDIR and WELLKNOWN variables are exported and can be used in an external program # default: HOOK=${BASEDIR}/hook.sh # Chain clean_challenge|deploy_challenge arguments together into one hook call per certificate (default: no) HOOK_CHAIN="no" # Minimum days before expiration to automatically renew certificate (default: 30) #RENEW_DAYS="30" # Regenerate private keys instead of just signing new certificates on renewal (default: yes) #PRIVATE_KEY_RENEW="yes" # Create an extra private key for rollover (default: no) #PRIVATE_KEY_ROLLOVER="no" # Which public key algorithm should be used? Supported: rsa, prime256v1 and secp384r1 #KEY_ALGO=rsa # E-mail to use during the registration (default: ) CONTACT_EMAIL=myemail@mydomain.com # Lockfile location, to prevent concurrent access (default: $BASEDIR/lock) #LOCKFILE="${BASEDIR}/lock" # Option to add CSR-flag indicating OCSP stapling to be mandatory (default: no) #OCSP_MUST_STAPLE="no" # Fetch OCSP responses (default: no) #OCSP_FETCH="no" # Issuer chain cache directory (default: $BASEDIR/chains) #CHAINCACHE="${BASEDIR}/chains" # Automatic cleanup (default: no) AUTO_CLEANUP="yes" - Created a new file with the list of domains to register:
nano ~/domains.txt
The file looks like this (obviously, put your own data in there):
myhost.mydomain.com
- Checked that the certificate registers correctly – note that if you are having trouble, you should enable the “staging” CA / terms file in config while you troubleshoot to avoid hitting letsencrypt limits:
dehydrated@localhost:~$ ./dehydrated --cron --accept-terms --out /etc/letsencrypt # INFO: Using main config file /home/dehydrated/config + Generating account key... + Registering account key with ACME server... Processing myhost.mydomain.com + Signing domains... + Creating new directory /home/dehydrated/certs/myhost.mydomain.com ... + Generating private key... + Generating signing request... + Requesting challenge for myhost.mydomain.com... Creating challenge record for myhost.mydomain.com in zone cynexia.net Created record: '_acme-challenge.myhost.mydomain.com. 60 IN TXT "cpE1VF_xshMm1IVY1Y66Kk9Zb_7jA2VFkP65WuNgu3Q"' Waiting for sync................................... Completed + Responding to challenge for myhost.mydomain.com... Deleting challenge record for myhost.mydomain.com from zone mydomain.com 1 record sets deleted + Challenge is valid! + Requesting certificate... + Checking certificate... + Done! + Creating fullchain.pem... + Using cached chain! + Done! + Running automatic cleanup
- Created the helper script, remembering to edit the domain (find/replace) and the certificate path:
nano updateunificert
My file looks like this:!/bin/bash PATH='/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games'; openssl pkcs12 -export -in /etc/letsencrypt/myhost.mydomain.com/fullchain.pem -inkey /etc/letsencrypt/myhost.mydomain.com/privkey.pem -out /etc/letsencrypt/myhost.mydomain.com/cert_and_key.p12 -name tomcat -CAfile /etc/letsencrypt/myhost.mydomain.com/chain.pem -caname root -password pass:aaa; rm -f /etc/letsencrypt/myhost.mydomain.com/keystore; keytool -importkeystore -srcstorepass aaa -deststorepass aircontrolenterprise -destkeypass aircontrolenterprise -srckeystore /etc/letsencrypt/myhost.mydomain.com/cert_and_key.p12 -srcstoretype PKCS12 -alias tomcat -keystore /etc/letsencrypt/myhost.mydomain.com/keystore; keytool -import -trustcacerts -alias unifi -deststorepass aircontrolenterprise -file /etc/letsencrypt/myhost.mydomain.com/chain.pem -noprompt -keystore /etc/letsencrypt/myhost.mydomain.com/keystore; mv /var/lib/unifi/keystore /var/lib/unifi/keystore-backup; cp /etc/letsencrypt/myhost.mydomain.com/keystore /var/lib/unifi/keystore; service unifi restart;
- Created cron jobs to trigger the cert update daily:
crontab -e
I added this line:20 5 * * * /home/dehydrated/dehydrated --cron --accept-terms --out /etc/letsencrypt >/dev/null 2>&1
- returned to my previous shell:
exit - Created cron jobs to install the new cert daily:
sudo crontab -e
I added this line:29 5 * * * /home/dehydrated/updateunificert >/dev/null 2>&1
How to upgrade VMWare ESXi on HP Gen8 Microserver
- go to VMware ESXi Patch Tracker and check for the latest ImageProfile e.g.
ESXi-6.5.0-20170404001-standard - Shut down all VMs and turn on maintenance mode
- allow outbound firewall requests
esxcli network firewall ruleset set -e true -r httpClient
- execute the update:
esxcli software profile update -p ESXi-6.5.0-20170404001-standard -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml
- Disable firewall ports
esxcli network firewall ruleset set -e false -r httpClient
- Disable maintenance mode
- Reboot
Fix: /var/lib/docker/aufs/diff is consuming entire drive
Some of my docker containers were complaining that they didn’t have enough drive space. This looked odd – so i logged in to the host and checked around:
robert@d:/$ df -h Filesystem Size Used Avail Use% Mounted on udev 2.0G 0 2.0G 0% /dev tmpfs 394M 12M 382M 3% /run /dev/sda1 26G 25G 0 100% / tmpfs 2.0G 2.0M 2.0G 1% /dev/shm tmpfs 5.0M 0 5.0M 0% /run/lock tmpfs 2.0G 0 2.0G 0% /sys/fs/cgroup //fs.cynexia.net/largeappdata 2.7T 285G 2.4T 11% /mnt/largeappdata //fs.cynexia.net/video 5.4T 3.0T 2.4T 56% /mnt/video //fs.cynexia.net/appdata 2.5T 52G 2.4T 3% /mnt/appdata
All space used up. Huh. Wonder why? I did a quick check to see what’s using most space:
robert@d:/$ sudo du -xh / | grep '[0-9\.]\+G' 8.0K /var/lib/docker/aufs/diff/1dba1b90260105df03d0147c535c104cca0dd24fcc9273f0bc27b725c7cc676f/usr/local/crashplan/jre/lib/locale/zh.GBK/LC_MESSAGES 12K /var/lib/docker/aufs/diff/1dba1b90260105df03d0147c535c104cca0dd24fcc9273f0bc27b725c7cc676f/usr/local/crashplan/jre/lib/locale/zh.GBK 19G /var/lib/docker/aufs/diff 19G /var/lib/docker/aufs 2.2G /var/lib/docker/containers/8c5725f63f681e012fcc479e78133f31ab1c760b7d8d4e0a7e150d213face41f 2.3G /var/lib/docker/containers 21G /var/lib/docker 21G /var/lib 22G /var 25G /
clearly /var/lib/docker/aufs/diff is what’s causing it. Let’s clean that up:
robert@d:/$ docker rmi $(docker images -aq --filter=dangling=true) Untagged: mnbf9rca/getiplayer@sha256:9846b7570b5ba6d686be21623446cec8abd9db04cf55a39ce45cabfaa0d63f9f Deleted: sha256:011bf974552570c536f8f98c73e0ed7d09ef9e2bfcbc7b3f3e02e19682b7480e Deleted: sha256:a0637dd0588be6aee9f4655260176e6da802fcd92347cdf789ae84f3503322c3 Deleted: sha256:6e21a0999ad14a1cc0ccc8e31611b137793e3614338e01f920e13bfeb4128fdc Deleted: sha256:b98c7813439119c3d2f859060fe11bf10151f69587f850a48448cae0fa4d9305 Untagged: mnbf9rca/getiplayer@sha256:ad493202d196dfae418769428ba6dea4d576ce1adec7ebe90837d0b965fe9b42 Deleted: sha256:b8df5a1ffa1eedd7be03d4a2a37549bf81699cc6fa1586c1d3510d90d4e9e562 ... Deleted: sha256:07c09e3cb65b3cec786933f882a08d5b0a34cd94f6922ada0d6f0cf779482ee0
Let’s check now…
robert@d:/$ sudo du -xh / | grep '[0-9\.]\+G' 8.0K /var/lib/docker/aufs/diff/1dba1b90260105df03d0147c535c104cca0dd24fcc9273f0bc27b725c7cc676f/usr/local/crashplan/jre/lib/locale/zh.GBK/LC_MESSAGES 12K /var/lib/docker/aufs/diff/1dba1b90260105df03d0147c535c104cca0dd24fcc9273f0bc27b725c7cc676f/usr/local/crashplan/jre/lib/locale/zh.GBK 4.8G /var/lib/docker/aufs/diff 4.8G /var/lib/docker/aufs 2.2G /var/lib/docker/containers/8c5725f63f681e012fcc479e78133f31ab1c760b7d8d4e0a7e150d213face41f 2.3G /var/lib/docker/containers 7.4G /var/lib/docker 7.6G /var/lib 8.3G /var 11G /
much better! It turns out there are a few great cleanup agents e.g. docker-gc-cron which will do the job for me.