I was restarting a Docker container in my NAS’ DiskStation and I suddenly got a warning that my primary volume is mounted in read-only mode.

Checking dmesg, I saw an error about a corrupted leaf. At this point, I didn’t really know how Btrfs works, or what a leaf is.

[ 363.524916] BTRFS critical (device dm-1): [cannot fix] corrupt leaf: root=1461 block=8947565723648 slot=1, bad key order
[ 363.526807] md3: [Self Heal] Retry sector [229802368] round [1/2] start: sh-sector [76600704], d-disk [3:sata3p5], p-disk [0:sata1p5], q-disk [-1: null]
[ 363.529030] md3: [Self Heal] Retry sector [229802376] round [1/2] start: sh-sector [76600712], d-disk [3:sata3p5], p-disk [0:sata1p5], q-disk [-1: null]
[ 363.529228] md3: [Self Heal] Retry sector [229802368] round [1/2] choose d-disk
[ 363.529230] md3: [Self Heal] Retry sector [229802368] round [1/2] finished: get same result, retry next round
[ 363.529232] md3: [Self Heal] Retry sector [229802368] round [2/2] start: sh-sector [76600704], d-disk [3:sata3p5], p-disk [0:sata1p5], q-disk [-1: null]
[ 363.529391] md3: [Self Heal] Retry sector [229802368] round [2/2] choose p-disk
[ 363.529394] md3: [Self Heal] Retry sector [229802368] round [2/2] finished: get same result, give up
[ 363.538846] md3: [Self Heal] Retry sector [229802384] round [1/2] start: sh-sector [76600720], d-disk [3:sata3p5], p-disk [0:sata1p5], q-disk [-1: null]
[ 363.539030] md3: [Self Heal] Retry sector [229802376] round [1/2] choose d-disk
[ 363.539032] md3: [Self Heal] Retry sector [229802376] round [1/2] finished: get same result, retry next round
[ 363.539035] md3: [Self Heal] Retry sector [229802376] round [2/2] start: sh-sector [76600712], d-disk [3:sata3p5], p-disk [0:sata1p5], q-disk [-1: null]
[ 363.539187] md3: [Self Heal] Retry sector [229802376] round [2/2] choose p-disk
[ 363.539190] md3: [Self Heal] Retry sector [229802376] round [2/2] finished: get same result, give up
[ 363.549362] md3: [Self Heal] Retry sector [229802392] round [1/2] start: sh-sector [76600728], d-disk [3:sata3p5], p-disk [0:sata1p5], q-disk [-1: null]
[ 363.549567] md3: [Self Heal] Retry sector [229802384] round [1/2] choose d-disk
[ 363.549570] md3: [Self Heal] Retry sector [229802384] round [1/2] finished: get same result, retry next round
[ 363.549572] md3: [Self Heal] Retry sector [229802384] round [2/2] start: sh-sector [76600720], d-disk [3:sata3p5], p-disk [0:sata1p5], q-disk [-1: null]
[ 363.549738] md3: [Self Heal] Retry sector [229802384] round [2/2] choose p-disk
[ 363.549741] md3: [Self Heal] Retry sector [229802384] round [2/2] finished: get same result, give up
[ 363.559460] md3: [Self Heal] Retry sector [229802392] round [1/2] choose d-disk
[ 363.560726] md3: [Self Heal] Retry sector [229802392] round [1/2] finished: get same result, retry next round
[ 363.562301] md3: [Self Heal] Retry sector [229802392] round [2/2] start: sh-sector [76600728], d-disk [3:sata3p5], p-disk [0:sata1p5], q-disk [-1: null]
[ 363.564761] md3: [Self Heal] Retry sector [229802392] round [2/2] choose p-disk
[ 363.566015] md3: [Self Heal] Retry sector [229802392] round [2/2] finished: get same result, give up

I spent two days trying to recover. Most of the advice is to salvage the files and rebuild the filesystem.

Potential cause is bitflip in memory. I recently upgraded my RAM to 16GB × 4. I did not test it and just plugged it in. After a couple of days, my filesystem got corrupted.

List btrfs devices

btrfs fi show

Unmount volume and stop services in DSM

synostgvolume --unmount /volume2
# I forgot the correct command, but it should resemble something like --unmount-with-packages

This is supposed to stop services and unmount the volume, but it was not working for me.

Trying out btrfs check --repair

I tried btrfs check --repair as a last resort. But I’m blocked by the following error. I was not able to figure out how to fix it.

couldn't open RDWR because of unsupported option features (800000000000003).

Mounting DSM volumes in Ubuntu

apt-get update
apt-get install -y mdadm lvm2  # Initiate mdadm
mdadm -AsfR                    # Assemble or activate an array, scan for MD superblocks, etc.
vgchange -ay                   # Activate volume group
cat /proc/mdstat               # List active RAID arrays
btrfs fi show                  # List btrfs devices
btrfs check /dev/mapper/vg1000-lv

Attempted to mount the volume in Ubuntu because I could not unmount it in DSM. I could not do anything aside from btrfs check because of an unknown feature error. I’m thinking DSM have custom code baked into their Btrfs bundle.

couldn't open RDWR because of unsupported option features (0x800000000000003)
ERROR: cannot open file system

Summary

I decided to back up everything and rebuild my volume. It was originally built last July 2020 and has gone through a lot of changes such as disk size increase (adding new disks). There were a lot of errors in btrfs check too.

It’s hard to continue using it with doubt if the error will not happen again.

The bad key order corruption was likely to be from memory bitflip. I’ll do a memtest on the host machine before doing anything else.

Cutting my losses by not spending more time on this issue. I learned a bit about Btrfs which is good because I will still use it. I have a better idea next time what to check.

Writing this down so I have reference in the future.

Resources

• https://gist-github-com.translate.goog/bruvv/d9edd4ad6d5548b724d44896abfd9f3f?_x_tr_sl=auto&_x_tr_tl=en&_x_tr_hl=en&_x_tr_pto=wapp
• https://btrfs.readthedocs.io/en/latest/btrfs-check.html
• https://btrfs.readthedocs.io/en/latest/btrfs-inspect-internal.html
• https://btrfs.readthedocs.io/en/latest/btrfs-rescue.html
• https://github.com/kdave/btrfs-progs/blob/master/check/repair.c
• https://kb.synology.com/tr-tr/DSM/tutorial/How_can_I_recover_data_from_my_DiskStation_using_a_PC
• https://github.com/jmiller0/how-to-fsck-on-synology-dsm7?tab=readme-ov-file

I had an idea to emulate an NVME and try to use it as cache in Synology DSM. I get to a point where it tried to mount the cache, then I get a segfault.

[ 1846.792660] kvm[24147]: segfault at 0 ip 000055bb2d97fb32 sp 00007fc7a62a2fb0 error 4 in qemu-system-x86_64[55bb2d857000+613000] likely on CPU 1 (core 1, socket 0)
[ 1846.793173] Code: e1 27 54 00 e8 6f 7b ed ff 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 53 48 8b 77 30 48 89 fb 44 8b 43 38 48 8b 06 8b 7e 60 <48> 8b 08 45 85 c0 78 46 80 7b 4c 00 74 58 8b 43 48 83 c0 01 3d 00
[ 1846.801440] zd64: p1 p2 p3

Proxmox does not yet support adding NVME drives using the web interface. It has to be added using a custom args for QEMU.

To do so, open the VM config file in a text editor: vi /etc/pve/qemu-server/100.conf

args: -drive file=/dev/zvol/rpool/data/vm-100-disk-1,if=none,id=nvme1 -device nvme,drive=nvme1,serial=nvme1

file is the location of the image in the hard drive. Since I’m using ZFS, it is in /dev/zvol/rpool/data/vm-100-disk-1.

I created this image using the Web UI but it is unattached. It is created in a real NVME disk.

device is how it will show-up in the guest.

QEMU docs on NVME emulation: https://www.qemu.org/docs/master/system/devices/nvme.html

Patch for Proxmox NVME emulation that has not yet been merged: https://bugzilla.proxmox.com/show_bug.cgi?id=2255

This was the state of my homelab. Whenever I go at my server room and attempt to arrange things, I get paralyzed how to get started.

Primary blocking reason is how I should handle supplying power. Currently I have 2 separate line-interactive UPS. One for critical internet-related devices, the other for optional servers.

My original plan was to make a DIY UPS using devices for solar. I “thought” it was cheaper. I planned to use:

• ATS (automatic transfer switch) if I have to have a maintenance, servers will keep running
• SNAT 1kw inverter - act like a line-interactive UPS
• Used LifePO4 battery
• Rack DIN rail for safety devices - ATS, breakers, DIN power plug

I already bought the components but assembling everything was daunting. I had a safety concern mixing high-voltage devices to the same rack as the server. The rack DIN rails would also have exposed live wires and I wasn’t comfortable with the plan. I purchased before I thought through the plan.

I was in a limbo on how to proceed. My homelab has been in this state for more than a year.

Finding a reasonably-priced On-line UPS

I finally found a reasonably priced online UPS. It’s on-line meaning it has double conversion 230v AC -> 24v DC -> 230v AC. Any power fluctuation won’t reach the devices.

It’s KSTAR YDC9101S RT. It’s only 900w as I don’t intend to run anything more than that. It has a user replaceable battery. It can be replaced while plugged-in. And it can be rack-mounted!

Cost is 10k, including shipping. This is cheap as compared to other on-line UPS I checked. Second-hand UPS without battery is easily around 20k. So I bit the bullet and purchased it.

A proper UPS, finally.

Plan of action

I knew I’ll rabbit-hole to something else when I start working on my homelab. There’s always something else to do. What I did was list what I wanted to accomplish for the day and limit myself to 2 hours.

1. Remove 2 line-interactive UPS
2. Remove shoe rack
3. Remove TP-Link 16-port Swtich
4. Install KSTAR UPS
5. Install 10GBe Netgear Switch
6. Install rack drawer

With a clear plan of action, I started.

Removing deprecated stuff

• Shoe rack - I used this as a poor man’s rack. It become unmanageable quickly
• 16-port TP Link switch - turns out I don’t use more than 8 ports anymore. With a new 10Gbe switch, it’s more than enough
• 2 extension cord
• 2 UPS with modified batteries

Installing new stuff

• KStar On-line UPS (bottom)
• Netgear 10Gbe Switch
• Rack sliding cabinet - I felt like I needed this to put loose items such as USB keyboard and mouse, extra SFP modules, rack screws. I had instances where I spent an afternoon looking for those.

Finished everything in 2 hours. Listing out what I want to accomplish was handy. It keep me on-track. I feel good about the progress I made with my homelab.

I stumbled upon a video that upgrades an iPhone 6s from 16gb to 128gb a couple of years back. It was super fascinating to learn that it was possible at all. It remained at the back of my head.

Years later (this year), I finally pulled the trigger to try it out when I found out that tools needed to do it are relatively cheap, and a broken 8-year-old iPhone 6s are cheap too. I have zero soldering skills, let alone microsoldering. But I just had to try it.

First attempt: broke the PCB pads

The NAND IC is the actual storage and what needs to be replaced to upgrade. What I did not expect was it’s soldering back the IC that’s the easy part. Removing the factory-soldered IC is hard because aside from the solder itself sticking to the motherboard, there’s an underfill that acts as an adhesive too.

I’m not supposed to put force pulling it out, but I did not know nor have the experience to know which is which.

If the PCB pads are broken, it’s connection to wherever it’s supposed to connect is gone.

Giving up: maybe this is not for me

I burned through 3 iPhone 6s at this point and I kept doing the same mistake of breaking the PCB pads. I’m losing hope and very frustrated and started questioning why I’m doing this at all.

Luckily, on my 4th attempt, the pads I’ve broke turned out to be “redundant” pads. It means as long as at least one can make a good connection, it will still work.

Super happy when I learned this!

Reballing and putting back the NAND

To put back the NAND IC, it needs to be reballed. Reballing is putting back the solder on it’s connection points that will be then used to connect back to the motherboard.

I used a medium-temperature solder paste with BGA70 stencil. This turned out to be easy and fun to do.

Putting it back is as easy as placing it in the correct orientation and heating it up. There will be a small movement as it place itself when the solder melt.

Finally made it work!

All these happened in span of weeks. For every failed attempt, I had to source another cheap broken iPhone 6s. I can only try again if I have another phone available. So there’s a lot of pent-up frustration.

I haven’t been able to clean up one of my tables because I haven’t finished this hobby yet. All the parts are scattered around, and it’s still a work in progress.

So when I finally made it, I felt so relieved. I can finally move on.

Things I learned:

• Previously, my approach to malfunctioning electronics was simple: if it didn’t power up, the whole component has to be replaced. Board-level repair was not even considered. Now, I’ve become more comfortable with electronics, and I’ve gained a better understanding of how they work.
• Electronic components can surprisingly withstand a lot of heat. My hot air workstation is set at 360C to remove and put back components, and it still work!
• It’s hard to do microsoldering without a microscope. But a microscope’s price is hard to justify for something I do not do professionally. It costs around 14k. So I tried to make do with my 20/20 vision and my phone’s macro lens.

I could not see this helping me with my professional career at all. This is just one of those passionate, dedication to the pursuit of short-term goals. This is just me indulging my curiosity.

I can finally clean up my dining table.

When I learned the term about multihoming, I revisited my interest again to have a VPS (Virtual Private Server) to setup as a VPN (Virtual Private Network) for my home network.

I wanted it to be hosted in the Philippines because of latency. Major hosting providers uses Singapore or Hong Kong to cater the PH market. But the latency is just too high at around 30ms.

I found one in LightNode. The cost is reasonable too for $7.7 USD/month.

The drawback is it looks to be limited to 100mbps only. Which is still reasonable.

Checking the hops, it looks like they are hosted within PLDT’s data center:

This is why the latency is at a very good range of 7ms. It does hop at one NAT which I think adds the 2ms. Overall this is good.

I’m going ahead and continue to use this and integrate it with my home network setup.

Next step is to understand multihoming.

One of the hobbies I picked up this year was to be an Internet Service Provider (ISP) wannabe using Fiber to the Home (FTTH).

I was just so fascinated with FTTH technology and how accessible acquiring the equipment needed to implement it. It’s expensive, but not out-of-reach expensive. Imagine the jump of technology from CAT6 (4 pairs of copper wires) limited to 100 meters, to a single strand of fiber than can support over 100 km length at faster speed! It’s super fascinating for me. 

I kept researching how it works. Knowing myself, I will only learn if I put skin in the game.

In the process, I learned a ton of new acronyms:

OLT: Optical Line Terminal

This is the most expensive part of this hobby. OLT is like the network switch for fiber. It allows transmission to multiple ONU (more on this later).

I chose the cheapest OLT I could find, specifically looking for a local supplier so I can quickly get a replacement if ever it malfunction.

I got a HiOSO HA7302CST. It’s an EPON OLT that has 2 PON ports that can connect up to 128 ONUs.

PON: Passive Optical Network

PON is what the technology is called. It’s passive, meaning it does not require electrical components to split or combine the signal. It literally uses light to transmit data.

There are 2 common types of implementation. EPON and GPON. EPON has symmetrical speed, GPON have faster downstream speed. GPON is what PLDT, Globe use as it has more capacity per fiber line. EPON is generally cheaper.

ODN: Optical Distribution Network

ODN is how the fiber are physically laid out.

We have a couple of properties in the same village. What I did is I installed a 1:8 NAP on each property to provide internet service to the house itself and it’s nearby neighbors.

NAP: Network Access Point

NAP are the boxes where the signal from PON is split. It can be split on different ratios with different signal loss depending on how large the split is.

Splitter Type	Insertion Loss (dB)
1:2	4.0
1:4	7.4
1:8	10.5
1:16	13.5

I opted to use 1:8, since I’m not seeing myself acquiring a lot of subscribers.

ONU: Optical Network Unit

ONU are the modem that converts the signal from fiber and make it available through Ethernet and WiFi.

I learned that Huawei EchoLife HG8145V5 is very popular modem in Facebook Marketplace because it’s what the major telecom uses. Unused modems from disconnected/terminated subscribers are being sold in the second-hand market.

It’s possible to reuse it outside the telecom’s network because there’s an exploit for Huawei modems that allows root access to the device and convert it from GPON to EPON ONU. This was fascinating too!

SC-UPC / SC-APC: Subscriber Connector

There are two types of subscriber connectors. SC-UPC (Blue) and SC-APC (Green). The difference is how their end is polished (angled vs not angled). It can be interchangeably used, but it will incur a notable signal loss.

I learned how to terminate it myself. So much easier than terminating a CAT6 cable.

Loss Budgeting

From the OLT, you start with the PON module signal which in my case is 7-9 dBm depending on the module. The loss must not to be more than -27 dBm when it reaches the ONU.

For every split and every termination, there will be signal loss incurred.

A computation is required to get an estimate of the final signal strength. I learned that it’s called loss budgeting — making sure that the signal does not go below the threshold from OLT to ONU.

-

I’m now an ISP for a couple of houses in village. It’s mostly family and friends. No plans on expanding, just wanted to scratch an itch and learn about the thing. Fun stuff!

My Mom asked for a CCTV at their place. There were disputes were a CCTV would have been beneficial to quickly sort out the issue.

I initially setup the “easy” kind of CCTV. An IoT WiFi-based camera, but it turned out not to be reliable. It saves its data on a MicroSD, and it easily burn out after a couple of months, wireless connection gets disconnected intermittently, and the camera itself is unpredictable and hangs from time to time.

I switched to PoE-based solution which is far better that what I initially installed.

Network Video Recorder

With PoE Camera, it does not have it’s own storage. The data has to be stored somewhere, which is the NVR (Network Video Recorder)

I have an existing server for Crystaldrin Network which serves various purposes including a NAS (Network Attached Storage). I was looking for something I could virtualize and use the same storage already available.

I explored a couple of software solutions and I decided on Synology’s Surveillance Station. The good things is, it’s Linux-based, and there’s a community that allows it to run in a virtualized environment: https://xpenology.com/forum/

I chose to emulate DVA3119 since I had a unused Nvidia GTX 1060 3gb lying around which would allow me to use deep learning features such as facial recognition and object detection.

I have it running on my Proxmox server with the GPU passed-through to guest virtual machine.

The GPU gets successfully detected by the virtual machine after booting it up.

Object Detection

This is where things got exciting for me. I think the usefulness of a CCTV comes with the ease on how easy it is to find what you’re looking for.

Surveillance Station’s interface itself is very intuitive. It also have a mobile app so it can be accessed remotely.

What’s more is they added features that can detect objects (e.g. plate number, people, vehicles). With this info, you can filter by time and date and/or people and/or plate numbers.

Backup Power

To make it more enterprise-y, it has to be somehow resilient to power interruption. Since the camera are powered over ethernet, I only have to have a backup power for the PoE switch.

I used SNADI 1kw with a 12V 100Ah LifePO4 battery. Based on my computation, it should run around 6 hours without power from Meralco.

This was a fun project. I get a lot of satisfaction finding uses for old hardware. This project is definitely one of those.

I’ve been trialing GitHub Copilot for a couple of weeks now. I can say that it’s well worth it especially when writing code from a blank slate. It outputs code blocks that help my brain process more quickly.

Its output rarely works as-is, but it only needs minimal tweaking.

It’s another subscription though, and I try to limit my monthly expenses to a minimum. It’s a nice-to-have, but not a necessary-to-have with the amount of coding I do at the moment.

When I was migrating my old server, I saw in the Vultr Marketplace FauxPilot. It turned out it’s an open-source alternative to GitHub Copilot server using Salesforce CodeGen.

I just had to try it.

Requirements

The only physical requirement is an Nvidia GPU with CUDA support (I have a Tesla P4 ✅).

The rest are just software to install:

• Docker
• Nvidia Container Toolkit (Install Guide)
• curl and zstd

Server Installation

I use Ubuntu 20.04 as my base OS. From a fresh install, here’s what I needed to do to run FauxPilot.

1. Install Docker

curl https://get.docker.com | sh \
  && sudo systemctl --now enable docker

2. Install Nvidia Container Toolkit

distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \
  && curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey \
  | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
  && curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list \
  | sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' \
  | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

3. Install Nvidia CUDA drivers

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt-get update
sudo apt-get -y install --no-install-recommends cuda
sudo apt-get -y install nvidia-driver-535

Make sure the drivers are properly installed by running nvidia-smi.

4. Clone FauxPilot repo

git clone https://github.com/fauxpilot/fauxpilot.git
cd fauxpilot

5. Run FauxPilot setup

sudo bash ./setup.sh

I chose codegen-2B-multi because my GPU only has 8GB VRAM, and I’m coding in PHP. Higher parameter models require more VRAM and RAM.

6. Launch FauxPilot

sudo bash ./launch.sh

Demo video: /media/homelab/2023/08/Screen-Recording-2023-08-24-at-22.00.05.mov

Test if the server is working:

curl -s -H "Accept: application/json" \
  -H "Content-type: application/json" \
  -X POST \
  -d '{"prompt":"def hello","max_tokens":100,"temperature":0.1,"stop":["\n\n"]}' \
  http://localhost:5000/v1/engines/codegen/completions | jq

Response should be:

{
  "id": "cmpl-OCButmOAbNedOMOxjPc0v9skuLdk7",
  "model": "codegen",
  "object": "text_completion",
  "created": 1692885668,
  "choices": [
    {
      "text": "(self):\n return \"Hello World!\"",
      "index": 0,
      "finish_reason": "stop",
      "logprobs": null
    }
  ],
  "usage": {
    "completion_tokens": 11,
    "prompt_tokens": 2,
    "total_tokens": 13
  }
}

Client Setup

Now that I have a working server, I need to setup my client. There’s a VSCode extension called FauxPilot. The only configuration change needed is pointing to the server address. After that, it works right away.

Demo video:

The suggestions quality is very far from GitHub Copilot. But at least it works!

There are a lot of factors why it’s underperforming. Maybe it’s the model itself, or the size of the model I chose, limited context, difference in training data.

Regardless of the quality, it’s exciting that it can be ran locally with old techs.

Back in the day when I was regularly doing freelance website projects, I offer hosting it too. It was a good idea to setup my own and I did. cPanel fits the bill. It has a fixed monthly subscription, easy to use, and I can upgrade my server as I grow.

After a couple of years, it has switched to per user pricing. On top of it, I don’t work on freelance projects anymore. I’ve been paying $87.99/month out-of-pocket to maintain the server. It currently hosts 8 websites, which are all low-traffic and do not need the current capacity of the server.

Downgrading server and switching off cPanel

Downgrading and switching off has been on my list ever since cPanel started charging per user.

When I try to search for free alternatives, I get choice paralysis. At this point though, I’m seeing other good things I could spend the monthly cost of the server.

I can use the minimum server instance size + a cPanel alternative.

For the alternative, I was looking specifically at:

• No monthly additional cost (biggest gripe with cPanel)
• Ease of setup
• Ease of migration

I checked VirtualMin, Webmin, VestaCP, Cyber Panel. All satisfy the first 2 criteria. When I saw that Cyber Panel supports importing cPanel account, I went ahead and spun up an instance to try it.

Cyber Panel is available in Vultr Marketplace Apps, no installation process was needed.

Data migration

In cPanel, go to Backups and click Download a full account backup. If you choose to save in the home directory, a file will be available at: /home/cpanelusername/backup-8.27.2020_08-58-02_cpanelusername.tar.gz

wget <backup_url>

To make it easier for me, I moved the file to public_html and download it in the new server.

/usr/local/CyberCP/bin/python /usr/local/CyberCP/plogical/cPanelImporter.py --path /root/cpanel_backups/

This command will import all backup files inside the cpanel_backups folder.

I tried it on one account. Update the DNS record to point to the new server, and surprisingly it worked right away.

The WordPress database was imported as well. For the subsites (additional domains), I had to explicitly set the PHP version and it worked as-is.

With a pleasant experience, I got hooked to do it for the rest of the accounts.

Nameserver

My cPanel installation also serves as the nameserver for some of the domains I host. Which is a good thing because I did not have to contact each person to update their domain.

Good thing, Cyber Panel comes with its own DNS server too. The zone file are imported with an updated server IP address.

Once everything has been migrated, I only needed to update the DNS of the nameservers to point to the new Cyber Panel instance.

Shutting down the server

I turned off the server to make sure everything is still working. After a couple of hours, I decided it was time to destroy it to save cost. From $87.99, my monthly bill for the new server is $12.00. Around 86% cost reduction.

Thank you for the 6 years, server1.jerico.ph.

version: '3.5'
services:
  jellyfin:
    image: jellyfin/jellyfin
    container_name: jellyfin
    user: 1026:100
    group_add:
      - "937"
    network_mode: 'host'
    volumes:
      - /volume2/docker/jellyfin/config:/config
      - /volume2/docker/jellyfin/cache:/cache
      - /volume2/Media/:/media
    restart: 'unless-stopped'
    devices:
      - /dev/dri/renderD128:/dev/dri/renderD128
    # Optional - may be necessary for docker healthcheck to pass if running in host network mode
    extra_hosts:
      - "host.docker.internal:host-gateway"

To get group id of renderD128 device:

cat /etc/group | grep videodriver | cut -d: -f3

This will allow hardware accelerated encoding.

My UPS have a serial port to report operating data. I read in the manual that it uses Upsilon 2000 software to get data. It uses the Megatec Protocol which is supported by Network UPS Tool’s blazer_ser driver.

I learned this by running nut-scanner --eaton_serial /dev/ttyUSB0 which tries different protocols to the serial connection.

root@jco-tc-pm01:~# nut-scanner --eaton_serial /dev/ttyUSB0
SNMP library not found. SNMP search disabled.
Neon library not found. XML search disabled.
IPMI library not found. IPMI search disabled.
Scanning serial bus for Eaton devices.
[nutdev1] driver = "blazer_ser" port = "/dev/ttyUSB0"

Plug-in USB to RS232 connector. dmesg should yield something like:

[257226.785044] usb 1-2: new full-speed USB device number 2 using xhci_hcd
[257226.933739] usb 1-2: New USB device found, idVendor=0557, idProduct=2008, bcdDevice= 3.00
[257226.933743] usb 1-2: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[257226.933745] usb 1-2: Product: USB-Serial Controller D
[257226.933746] usb 1-2: Manufacturer: Prolific Technology Inc.
[257226.940963] usbcore: registered new interface driver usbserial_generic
[257226.940968] usbserial: USB Serial support registered for generic
[257226.941833] usbcore: registered new interface driver pl2303
[257226.941840] usbserial: USB Serial support registered for pl2303
[257226.941848] pl2303 1-2:1.0: pl2303 converter detected
[257226.942410] usb 1-2: pl2303 converter now attached to ttyUSB0

Access raw data

# Add current user to dialout group
usermod -a -G dialout <user> # Connect to serial using screen
screen /dev/ttyUSB0 2400 # Status query using Q1
Q1 # It would write back
(230.1 241.7 230.2 027 60.4 2.28 34.0 00000000 # 230.1 - Input voltage
# 241.1 - Fault voltage
# 230.2 - Output voltage
# 60.4 - Input Frequency
# 2.28 - Battery voltage
# 34 - Temperature

NUT

Config:

vim /etc/nut/nut.conf

MODE=netserver

vim /etc/nut/ups.conf

Add output of nut-scanner:

root@jco-tc-pm01:~# upsdrvctl start
Network UPS Tools - UPS driver controller 2.7.4
Network UPS Tools - Megatec/Q1 protocol serial driver 1.57 (2.7.4)
Supported UPS detected with megatec protocol
Vendor information read in 1 tries
No values provided for battery high/low voltages in ups.conf Using 'guestimation' (low: 20.800000, high: 26.000000)!
Battery runtime will not be calculated (runtimecal not set)

upsc nutdev1

battery.charge: 100
battery.voltage: 27.24
battery.voltage.high: 26.00
battery.voltage.low: 20.80
battery.voltage.nominal: 24.0
device.mfr: device.model: device.type: ups
driver.name: blazer_ser
driver.parameter.pollinterval: 2
driver.parameter.port: /dev/ttyUSB0
driver.parameter.synchronous: no
driver.version: 2.7.4
driver.version.internal: 1.57
input.current.nominal: 4.0
input.frequency: 60.4
input.frequency.nominal: 60
input.voltage: 230.4
input.voltage.fault: 241.7
input.voltage.nominal: 220
output.voltage: 230.1
ups.beeper.status: disabled
ups.delay.shutdown: 30
ups.delay.start: 180
ups.firmware: V022B000D0
ups.load: 30
ups.mfr: ups.model: ups.status: OL
ups.temperature: 34.0
ups.type: online

I’ve been wanting a solution to have conversation

For the past couple of days I’ve been experimenting with Wyze Cam v2 and Xiaomi Xiaofang 1s. It’s an identical product that has Ingenic T20 as SoC.

It’s a camera we used 3-4 years ago but stopped using because of general unreliability (microSD regularly burning out, occasional disconnection).

I really did not have any goal other than leisurely tinker around. It has a couple of custom firmware/firmware modification projects actively being developed I want to try first hand.

Hunch: WiFi is the cause of unreliability

I think the primary issue I had when I was using this was consistent disconnection that required a power cycle. Unreliability for a device that’s meant as a “security” camera made it undesirable.

My hunch is it’s because of the WiFi connection. It has a RTL8189 WiFi chip that’s using 2.4 Ghz only, which likely already noisy. Video data stream is high-bandwidth already running 24/7.

I had the idea of trying not using WiFi altogether.

There are a couple of ways to do it:

As a Webcam

Wyze has a webcam-only firmware which surprisingly works on both Wyze Cam v2 and Xiaomi Xiaofang 1S.

It shows up as a USB Video Class (Webcam), Mic, and Speaker.

The speaker does not sound good, but it works.

Doing it this way would require a PC to host the webcams, make the video stream available in the network via RTSP. Possibly using v4l2rtspserver or go2rtc.

Benefit is it removes the network and data processing off the camera, but it will be limited by USB cable length.

Problem: USB cable length

RJ45 extension does not work

I checked if there’s a USB to RJ45 to USB. RJ45 is very easy to extend and terminate. And there is:

Link: https://s.lazada.com.ph/s.7ZW19
Price: Php 138.00

Even with 0.5m CAT6a cable, the video signal does not work. The device shows up in the device list but the video is just black. I think it has to do with data integrity not being retained after passing through CAT6 cable.

To verify that it was not a defective adaptor, I tried a USB mouse which worked.

10 meter USB-A to USB-A worked

The longest USB-A to USB-A cable I found is 10 meters. It worked without issue.

Link: https://s.lazada.com.ph/s.7ZWZQ
Price: Php 139.00

Problem: USB root hub cannot support more than 3 cameras at the same time due to bandwidth constraint

I tried plugging in 2 camera and my Mac supported running it at the same time. Both seem to have a different Unique ID.

Reference: https://www.any-maze.com/support/guides/connecting-multiple-usb-cameras-to-a-computer/ As a rule of thumb, you can only connect three USB cameras to a single USB controller.

Video4Linux

Supported format includes H264, meaning no encoding needed. This might also mean that the camera will need less bandwidth because the stream is already compressed.

# v4l2-ctl -d /dev/video0 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT Type: Video Capture
[0]: 'MJPG' (Motion-JPEG, compressed)
  Size: Discrete 640x360
    Interval: Discrete 0.033s (30.000 fps)
  Size: Discrete 1280x720
    Interval: Discrete 0.033s (30.000 fps)
  Size: Discrete 1920x1080
    Interval: Discrete 0.033s (30.000 fps)
[1]: 'YUYV' (YUYV 4:2:2)
  Size: Discrete 640x360
    Interval: Discrete 0.033s (30.000 fps)
  Size: Discrete 1280x720
    Interval: Discrete 0.200s (5.000 fps)
  Size: Discrete 1920x1080
    Interval: Discrete 0.200s (5.000 fps)
[2]: 'H264' (H.264, compressed)
  Size: Discrete 640x360
    Interval: Discrete 0.033s (30.000 fps)
  Size: Discrete 1280x720
    Interval: Discrete 0.033s (30.000 fps)
  Size: Discrete 1920x1080
    Interval: Discrete 0.033s (30.000 fps)

Useful link on debugging UVC on Linux: https://gist.github.com/lucasw/85dc92c9f5146e9d5175a33b49ef4a90

USB Direct

This method makes the camera show up as a network device. Processing will still be in camera. It can be configured like connected to a network.

I have not tried this yet though as I could not wrap my head around the network settings needed.

Link: https://github.com/gtxaspec/wz_mini_hacks/blob/master/documentation/usb-direct.md

USB to Ethernet Adaptor

I have not tried this yet but it requires a USB to Ethernet Adaptor and a separate way to power up the device.

Firmwares

wz_mini_hacks

Root access

Building firmware for v2

Firmware for Wyze Cam v2 is not made available for download. There’s a script that unpacks the firmware and replace the kernel, then repacks it.

The automated part to download the firmware is not working anymore because Wyze removed older firmwares. It only supports v2 4.9.8.1002 and pan v1 4.10.8.1002. It could theoretically support other version if the offset where to write the modified kernel is set. I did not dig deeper how to do this.

openmiko

Implements it’s own code to expose Ingenic T20 video streams. Supports 1080p by default. Only works with Wyze Cam v2.

Link: https://github.com/openmiko/openmiko

Xiaom-Dafang-Hacks

Compatible with both Wyze Cam v2 and Xiaomi Xiaofang 1S.

Link: https://github.com/EliasKotlyar/Xiaomi-Dafang-Hacks

docker-wyze-bridge

Not really a firmware but a bridge to expose native Wyze streams as RTSP.

Not reliable.

Link: https://github.com/mrlt8/docker-wyze-bridge

Firmware mix-and-match

Since both cameras are the same hardware-wise, I tried mix and matching:

Firmware	Wyze Cam v2	Xiaomi Xiaofang 1S
Wyze Webcam Firmware	Yes	Yes
OpenMiko	Yes	No
Xiaomi-Dafang-Hacks	Yes	Yes
wz_mini_hacks	Yes	No
Official Wyze Firmware	Yes	No

Hardware Mods

Since I have a lot of these cameras, something will inevitably fail.

Interchangeable Camera Module

I have a Wyze cam v2 that has a blurry sensor. I tried replacing the whole camera module from a Xiaomi Xiaofang 1S. It worked.

Useful links

https://leo.leung.xyz/wiki/Wyze_Cam

iGPU and SATA controller is in its own group

In preparation for setting up and upgrading CRYSTALDRIN server to DSM 7.2, I tested out what’s the new process for creating an Xpenology VM in Proxmox.

I remember it being really complicated that involves a lot of commands.

With the recent development of interactive bootloaders, it was very easy now.

I continued with my Xpenology Experiments when I remember that there was an NVR that’s using an older version of Nvidia GPU: DVA3219. It’s officially paired with GTX 1050Ti which is the same generation of the GPU I have in hand.

I checked if any of the bootloaders have support with DVA3219 platform, arpl-i18n does under experimental platforms.

I quickly tried it up again. The installation had no issues. It detected the GPU just like with DVA3221. But I’m still not sure if any AI tasks will work.

I installed Surveillance Station, added 1 IP camera and setup AI tasks.

Facial Recognition works!

Object Detection works!

Intrusion Detection and People Counting did not work though.

Another interesting thing is I can run up to 3 AI tasks at the same time. Having AI tasks enabled directly reflects in GPU utilization.

Power Consumption

No AI tasks	~50w
Facial Recognition Only	~100w
Object Detection Only	~80w
Facial Recognition + Object Detection	~100w

Running this machine 24/7 would roughly translates to around 700 pesos per month (@ 10 pesos/kw)

Where to use: CRYSTALDIN-NAS

It’s fun that I found a use for this GPU. I got it for free but I really have no use for it.

I have a server running at CRYSTALDRIN that has a free x16 PCIe port. It’s running Proxmox. I can set up a virtual DVA3219 with GPU Passthrough and have this up and running.

I still have reservation with the monthly cost. An alternative is to use a Intel-based GPU which would only use around 20w (80% less, around 144 pesos per month).

Let’s see!

The special configuration required for the virtual machine that simulates the U disk:

1. Select the model as q35.
2. Enable SeaBIOS, or select UEFI and enter BIOS to turn off secure boot.
3. Cancel all Boot Order under Options.
4. Edit /etc/pve/qemu-server/101.conf.

Add the following line in (note to modify the mirror path):

args: -device 'nec-usb-xhci,id=usb-bus0,multifunction=on' -drive 'file=/path/to/arpl-sa6400.img,media=disk,format=raw,if=none,id=drive-disk-bootloader' -device 'usb-storage,bus=usb-bus0.0,port=1,drive=drive-disk-bootloader,id=usb-disk-bootloader,bootindex=999,removable=on'

Source: https://blog.jim.plus/blog/post/jim/synology-sa6400-with-i915

vi /etc/default/grub
Find the line with "GRUB_CMDLINE_LINUX_DEFAULT"
Add `intel_iommu=on` or `amd_iommu=on`
Save
update-grub

PCIe ACS Override

There are motherboards that bundle peripherals together (does not support ACS).

A workaround is to override PCIe ACS by adding pcie_acs_override=downstream,multifunction in GRUB_CMDLINE_LINUX_DEFAULT.

This flags the kernel that a device can be “isolated”. It might have security and stability issues.

References:

• [PATCH] pci: Enable overrides for missing ACS capabilities https://lkml.org/lkml/2013/5/30/513
• IOMMU Groups, inside and out http://vfio.blogspot.com/2014/08/iommu-groups-inside-and-out.html

It cannot passthrough the GPU

I have a DS918+ which only have 2 GbE port. I’m in the process of upgrading my local network to 10GbE.

At the same time, I don’t want to replace this pricey NAS. I found out that there’s an option to purchase a USB 3.0 2.5G network dongle.

2.5GE is relatively cheap too, less than Php 856.88 in Lazada.

Here’s the drivers:

https://github.com/bb-qq/r8152

Ugreen USB C 2.5

https://github.com/bb-qq/aqc111

5g Qnap

AME patch is needed if you want support for HEVC encoding/decoding.

1. Connect to SSH.
2. Create the patch (below).
3. Run sudo python ame.py.

import hashlib
import os

r = ['669066909066906690', 'B801000000', '30']
s = [(0x1F28, 0), (0x48F5, 1), (0x4921, 1), (0x4953, 1), (0x4975, 1), (0x9AC8, 2)]
prefix = '/var/packages/CodecPack/target/usr'
so = prefix + '/lib/libsynoame-license.so'

print("Patching")
with open(so, 'r+b') as fh:
    full = fh.read()
    if hashlib.md5(full).digest().hex() != 'fcc1084f4eadcf5855e6e8494fb79e23':
        print("MD5 mismatch")
        exit(1)
    for x in s:
        fh.seek(x[0] + 0x8000, 0)
        fh.write(bytes.fromhex(r[x[1]]))

lic = '/usr/syno/etc/license/data/ame/offline_license.json'
os.makedirs(os.path.dirname(lic), exist_ok=True)
with open(lic, 'w') as licf:
    licf.write('[{"appType": 14, "appName": "ame", "follow": ["device"], "server_time": 1666000000, "registered_at": 1651000000, "expireTime": 0, "status": "valid", "firstActTime": 1651000001, "extension_gid": null, "licenseCode": "0", "duration": 1576800000, "attribute": {"codec": "hevc", "type": "free"}, "licenseContent": 1}, {"appType": 14, "appName": "ame", "follow": ["device"], "server_time": 1666000000, "registered_at": 1651000000, "expireTime": 0, "status": "valid", "firstActTime": 1651000001, "extension_gid": null, "licenseCode": "0", "duration": 1576800000, "attribute": {"codec": "aac", "type": "free"}, "licenseContent": 1}]')

print("Checking whether patch is successful...")
ret = os.system(prefix + "/bin/synoame-bin-check-license")
if ret == 0:
    print("Successful, updating codecs...")
    os.system(prefix + "/bin/synoame-bin-auto-install-needed-codec")
    print("Done")
else:
    print(f"Patch is unsuccessful, retcode = {ret}")

7.2:

import hashlib
import os

r = ['669066909066906690', 'B801000000', '30']
s = [(0x3718, 0), (0x60A5, 1), (0x60D1, 1), (0x6111, 1), (0x6137, 1), (0xB5F0, 2)]
prefix = '/var/packages/CodecPack/target/usr'
so = prefix + '/lib/libsynoame-license.so'

print("Patching")
with open(so, 'r+b') as fh:
    full = fh.read()
    if hashlib.md5(full).digest().hex() != '09e3adeafe85b353c9427d93ef0185e9':
        print("MD5 mismatch")
        exit(1)
    for x in s:
        fh.seek(x[0] + 0x8000, 0)
        fh.write(bytes.fromhex(r[x[1]]))

lic = '/usr/syno/etc/license/data/ame/offline_license.json'
os.makedirs(os.path.dirname(lic), exist_ok=True)
with open(lic, 'w') as licf:
    licf.write('[{"attribute": {"codec": "hevc", "type": "free"}, "status": "valid", "extension_gid": null, "expireTime": 0, "appName": "ame", "follow": ["device"], "duration": 1576800000, "appType": 14, "licenseContent": 1, "registered_at": 1649315995, "server_time": 1685421618, "firstActTime": 1649315995, "licenseCode": "0"}, {"attribute": {"codec": "aac", "type": "free"}, "status": "valid", "extension_gid": null, "expireTime": 0, "appName": "ame", "follow": ["device"], "duration": 1576800000, "appType": 14, "licenseContent": 1, "registered_at": 1649315995, "server_time": 1685421618, "firstActTime": 1649315995, "licenseCode": "0"}]')

print("Checking whether patch is successful...")
ret = os.system(prefix + "/bin/synoame-bin-check-license")
if ret == 0:
    print("Successful, updating codecs...")
    os.system(prefix + "/bin/synoame-bin-auto-install-needed-codec")
    print("Done")
else:
    print(f"Patch is unsuccessful, retcode = {ret}")

https://xpenology.com/forum/topic/65643-ame-30-patcher/

This all started when I was looking if there’s a new way to “trick” DSM’s Surveillance Station to increase camera limit.

I stumbled a post to just use an DVA1622 (NVR-specific NAS) and I’ll automatically get 8 licenses without resorting to use an old Surveillance Station apk.

When I checked Xpenology forum, I saw there’s an active development to run DVA1622 loader.

I’ve read through the thread to get a sense on compatibility. I tried tinycore-redpill first because it’s what I’ve used a couple of years back.

I could not finish the setup process though because of “out of storage” error when building the boot loader. I saw there was an effort to make the bootloader more streamlined. Literally just a set of menus to select what you want to happen and it will do everything for you. I used ARPL for my second attempt.

Easy mode: ARPL

I created a boot disk with ARPL. This is my first time using ARPL. I’m used to type ./rploader.sh commands from tinycore-redpill. The experience with ARPL was just delightful. No messing around with different commands.

After selecting “Boot loader”, it just works.

It felt super exciting making this work. I remember my college days when we could not afford a Macbook, and being able boot MacOS on my Compaq laptop, and actually do work on it. It felt amazing.

DVA1622 and Intel GPU

I chose DVA1622 as my platform to have the 8 camera license built-in. However, once done with the initial setup, the license was 0.

It turns out the issue is wrong serial number pattern. ARPL I used was not updated to generate the correct serial pattern. I rebooted to ARPL again, updated the serial number and it worked.

AI tasks

The next thing I wanted to try if it will work are AI tasks. DVA1622 has a new feature that can do Facial Recognition and Video Analytics (intrusion detection, read license plates, etc). It uses the Intel iGPU.

It’s supposed to be for Intel HD Graphics 630. The machine I’m testing it on only has Intel HD Graphics 520, but it uses the same i915.ko driver.

It worked!

DVA3221 and Nvidia GTX 1060 3gb

DVA3221 is a hardware that has an Intel Atom CPU and GTX 1650 GPU. It allows more up to 12 AI tasks (Facial Recognition and Video Analytics).

I do not have GTX 1650, but I have a GTX 1060 lying around. I was curious if it will work because technically Synology uses the same Linux drivers for Nvidia as everybody else.

The installation was straightforward. Super surprised and excited that GTX 1060 showed up in control panel.

I setup Facial Recognition and Video Analytics task. I monitored GPU processes using nvidia-smi. I can see synofaced and synodvad processing. After a couple of tries, nothing was being detected.

TLDR: Despite having the GPU being detected, it’s not working with AI tasks.

NVENC will work though. For packages (e.g. Plex) that has the feature to enable NVENC, it will work.

AI Tasks Compatibility

I tried AI tasks on a couple more hardware I have. Here’s what I found out:

	Facial Recognition	Video Analytics
Intel HD Graphics 630	✅	✅
Intel HD Graphics 520	✅	✅
Intel HD Graphics 4200 (from G1840)	❌	❌
Intel HD Graphics 610 (from G4560)	❌	❌
Nvidia GTX 1060 3gb	❌	❌

Step-by-step guide to create a boot disk for booting Xpenology bootloaders.

1. Download the bootloader you intend to use.
2. Extract the .gz to .img.
3. Plug-in the USB flash drive.
4. In a Mac terminal, run diskutil list.
5. Note the disk number of of the USB flash drive in the list.
6. Run sudo dd if=arpl.img of=/dev/rdisk# bs=1m.
   • Replace # with the disk number.

Xpenology DSM bootloader list:

• https://github.com/pocopico/tinycore-redpill
• https://github.com/fbelavenuto/arpl/releases (not updated anymore)
  • https://github.com/wjz304/arpl-i18n/releases (updated fork of arpl)
• https://github.com/AuxXxilium/arc

This is the current state of my homelab. Whenever I go at my server room and attempt to arrange things, I get paralyzed how.

Primary blocking reason is how I should handle supplying power. Currently I have 2 separate line-interactive UPS. One for critical internet-related devices, the other for optional servers.

My original plan was to make a DIY UPS using devices for solar. I “thought” it was cheaper. I planned to use:

• ATS (automatic transfer switch) if I have to have a maintenance, servers will keep running
• SNAT 1kw inverter - act like a line-interactive UPS
• Used LifePO4 battery
• Rack DIN rail for safety devices - ATS, breakers, DIN power plug

I already bought the components but assembling everything was daunting. I had a safety concern mixing high-voltage devices to the same rack as the server. The rack DIN rails would also have exposed live wires and I wasn’t comfortable with the plan. I purchased before I thought through the plan.

I was in a limbo on how to proceed. My homelab has been in this state for more than a year.

Finding a reasonably-priced On-line UPS

I finally found a reasonably priced online UPS. It’s on-line meaning it has double conversion 230v AC -> 24v DC -> 230v AC. Any power fluctuation won’t reach the devices.

It’s KSTAR YDC9101S RT. It’s only 900w as I don’t intend to run anything more than that. It has a user replaceable battery. It can be replaced while plugged-in. And it can be rack-mounted!

Cost is 10k, including shipping. This is cheap as compared to other on-line UPS I checked. Second-hand UPS without battery is easily around 20k. So I bit the bullet and purchased it.

A proper UPS, finally.

Plan of action

I knew I’ll rabbit-hole to something else when I start working on my homelab. There’s always something else to do. What I did was list what I wanted to accomplish for the day and limit myself to 2 hours.

With a clear plan of action, I started.

Removing deprecated stuff

• Shoe rack - I used this as a poor man’s rack. It become unmanageable quickly
• 16-port TP Link switch - turns out I don’t use more than 8 ports anymore. With a new 10Gbe switch, it’s more than enough
• 2 extension cord
• 2 UPS with modified batteries

Installing new stuff

• KStar On-line UPS (bottom)
• Netgear 10Gbe Switch
• Rack sliding cabinet - I felt like I needed this to put loose items such as USB keyboard and mouse, extra SFP modules, rack screws. I had instances where I spent an afternoon looking for those.

Finished everything in 2 hours. Listing out what I want to accomplish was handy. It keep me on-track. I feel good about the progress I made with my homelab.

1. Install Docker.
2. Create a folder.
3. Create a plain text file named docker-compose.yml.
4. Copy the code below.
5. Run docker-compose up.

services:
  wordpress:
    image: wordpress:6.1.1-apache
    environment:
      WORDPRESS_DB_HOST: db
      WORDPRESS_DB_USER: wordpress
      WORDPRESS_DB_PASSWORD: password
      WORDPRESS_DB_NAME: wordpress
    volumes:
      - ./wordpress:/var/www/html
  db:
    image: mysql:5.7
    platform: linux/x86_64
    environment:
      MYSQL_DATABASE: wordpress
      MYSQL_USER: wordpress
      MYSQL_PASSWORD: password
      MYSQL_RANDOM_ROOT_PASSWORD: "1"
    volumes:
      - ./db:/var/lib/mysql
  tunnel:
    image: cloudflare/cloudflared
    restart: unless-stopped
    command: tunnel --url wordpress:80
    depends_on:
      - wordpress
      - db

I’ve been curious on getting my own ASN and IP address.

The process is to register and buy in your region’s internet registry. In my case, it’s APNIC. APNIC is more strict than other registries in terms of registration and fees.

At the very minimum, I need 500 AUD registration fee and 1180 annual fee to keep a /24 IPv4. Which is more than the money I want to spend. On top of the fees, it looks like only a business entity can register, which I don’t have.

Source: https://www.apnic.net/get-ip/get-ip-addresses-asn/

Is it possible for an individual to get his own ASN?

Browsing ASNs from the Philippines, I saw two curious ASN number that looks to be registered under an individuals name: AS200866 and AS200879.

Both are registered under RIPE which has a more lenient registration process and provides free IPv6 allocations.

I’ll go through the registration process and see how it turns out.

Maybe I could get my own ASN too.

Another setup to try. I have a bunch of various size disk that I do not know what’s the best setup for.

mergerfs combines different disk with different filesystem and show it as a single mount. Files are still flat. Disks can still be mounted individually.

SnapRAID can be used to setup a parity drive to allow recovery if one drive fails.

The idea is to combine the two technology. Mergerfs to combine disks + SnapRAID for data redundancy.

Something to checkout.

https://selfhostedhome.com/combining-different-sized-drives-with-mergerfs-and-snapraid/

https://perfectmediaserver.com/tech-stack/snapraid/

I have multiple, different size hard drives. It’s hard to plan how to utilize it with data safety in mind. Usually, hard drives need to be the same specification to use as RAID.

I’ve been seeing GlusterFS and I was curious if it fits my use-case.

My understanding is it’s a distributed file system that takes care of replication across hosts. So a setup with 3 different servers, with 3 (or more) different disk sizes should work. Increasing the cluster size means adding additional servers with more disks.

I am seriously considering this for my homelab. I don’t have time for it now though, but it’s on my list to try.

References:

• https://blog.programster.org/glusterfs-distributed-volumes-with-different-size-bricks
• https://docs.gluster.org/en/main/Administrator-Guide/GlusterFS-Introduction/

For the past couple of days (when I’m procrastinating), I’ve been attempting to make an XFX RX 460 work as a GPU passthrough on a macOS KVM.

Ready-made macOS VM

Flashing a different BIOS

# Display all adaptors detected
./amdvbflash -ai
# Get info about the rom file
./amdvbflash -biosfileinfo vbios.rom
# Save bios
./amdvbflash -s 0 original_rom
# Force flash a rom
./amdvbflash -f -p 0 ./original_rom

Where to get BIOS rom files

https://www.techpowerup.com/vgabios/

How to know which BIOS are compatible

Run GPU-Z. Take note of the memory type and size.

Setting Proxmox to use a rom file

Where are the rom files located

/usr/share/kvm/romfile.rom

How to use the romfile

vi /etc/pve/qemu

Add romfile=romfilename.rom to hostpci0.

Giving up

I think I’ve tried almost all BIOS compatible with my graphics card. I was not able to make it work.

Too much time has been spent. I could have bought a fully compatible one if I worked for the same amount of time I spent on this.

It was a good try, but I need to cut my losses (of time).

When I learned the term about multihoming, I revisited my interest again to have a VPS to setup as a VPN for my network.

I wanted it to be hosted in the Philippines because of latency. Major hosting providers uses Singapore or Hong Kong to cater the PH market. But the latency is just too high at around 30ms.

I found one in LightNode. The cost is reasonable too for $7.7 USD/month.

The drawback is it looks to be limited to 100mbps only. Which is still reasonable.

Checking the hops, it looks like they are hosted within PLDT’s data center:

This is why the latency is at a very good range of 7ms. It does hop at one NAT which I think adds the 2ms. Overall this is good.

I’m going ahead and continue to use this and integrate it with my home network setup.

Next step is to understand multihoming.

Wow. This app mindblowingingly easy to install and use.

This is for managing and consuming audiobooks. I actually gave up trying to manage my own audiobooks and subscribed to Audible instead. The issue is I only have phases on when I listen to audiobooks. It only recently started when I started driving everyday again to pickup my kids from school.

The issue I have with Audible is it’s a subscription. And I don’t really own books I purchased there. It’s only rented until they close down.

With that concern, I’ve attempted to download copies of my books. But managing it as a flat file is hard. I have a folder “Audiobooks” where my audiobooks are left to be ignored.

Ang hirap kasi i-navigate. Opening it with the Music app, it will import the whole thing. Without context too, walang chapters, walang metadata.

This app solves all those. Grabe. The interface is super delightful to use to. As soon as I upload 1 book, I started listening. That’s it.

Wow.

Bigla kong na-miss yung Google Reader. I used to subscribe to blogs. Madami pa ring blogs ngayon but it’s hard to keep track unless may aggregator.

I started looking for self-hosted solution na may similar user experience with Google Reader. Yung naka-open na yung articles but scroll to read lang. FreshRSS fits the bill.

I’ve been meaning to set it up kasi may Docker package naman. I thought it will only take less than 30 minutes pero it’s 1.5 hours bago ko napaganda. It’s mix of where my containers are hosted, yung provided nilang default config na hindi gumagana out-of-the-box, and small changes sa config na I would not thought will break my initial installation.

But now it works:

Here’s to more curated way to consume content from the web!

Spent an hour of my morning setting up ArchiveBox. I’ve set this up before but I forgot how to access it. It wasn’t even running.

I saw that there was a docker-compose.yml already and I went to update Docker Compose in my Synology NAS and ran docker-composer up -d and it worked as it was before:

Why?

I blocked Reddit and other news site to my main computer to prevent me to rabbit-hole on unintended topics. I wanted some sort of deterrent when I impulsively type or visit a Reddit link.

I was watching a podcast and they linked to an interesting Reddit post about someone posting their net worth, growth, and plans. I did not want to unblock Reddit “temporarily”. I remembered this is a good use-case of ArchiveBox. I’ll archive the Reddit link and I’ll have access only to the archived page.

It worked:

<code>sudo su cd /var/packages/Docker/target/usr/bin/ mv docker-compose docker-compose_bak curl -L https://github.com/docker/compose/releases/download/v2.15.1/docker-compose-`uname -s`-`uname -m` -o docker-compose chmod +x ./docker-compose
</code>

Utilize/get more value out of Synology

Docker on Synology DSM

I’ve been considering changing my router from Omada’s R605 to something more configurable.

Omada is great for managing switch and access points. Sobrang seemless mag-add ng bagong device. Routing-wise, it works naman pero biggest gripe ko yung detection ng WAN connection. It can only do it per minute.

Kaya I’ve been looking to switch to Mikrotik CHR.

I found this blog post with benchmarks. VyOS came on top. I was not even aware of VyOS before. Kaya bagong rabbit hole nanaman. I’ll give it a shot soon.

VyOS

It supports PPPoE which is the main reason why I’m looking to switch sa Mikrotik.

https://docs.vyos.io/en/stable/configuration/service/pppoe-server.html

I have quite a few mini-PC that has only a single gigabit port. I’ve been considering and researching how to expand those to have an extra port. One of those options was using a USB to Ethernet adaptor.

Now, I’ve looked at this option before. Ang nakuha ko lang puro discouragement na hindi siya stable.

Why it’s not recommended

• Doesn’t offload all processing from the CPU, causing high CPU usage.
• Additional USB abstraction compared to PCIe.

I found this thread, they discussed different chipsets and they benchmarked it too: https://forums.macrumors.com/threads/macbook-air-usb-c-ethernet-unreliable.2287743/

CDC - Communication Device Class (USB)
NCM - Network Control Model
https://www.keil.com/pack/doc/mw/USB/html/group__usbd__cdc_functions__ncm.html

A CDC NCM compliant device exposes itself as a virtual NIC to the host operating system.

A CDC ECM is a predecessor of NCM that needs software implementation of other Ethernet standards, causing high CPU usage during transfer.

MacOS

AX88179A

AX88179A:
Bus: USB
Vendor Name: ASIX
Product Name: AX88179A
Vendor ID: 0x0b95
Product ID: 0x1790
USB Link Speed: Up to 5 Gb/s
Driver: com.apple.driver.usb.cdc.ncm
BSD Device Name: en4
MAC Address: 20:7b:d2:11:a4:e3
AVB Support: No
Maximum Link Speed: 2.5 Gb/s

• CPU heavy without driver on Mac. Driver: https://forums.macrumors.com/threads/macbook-air-usb-c-ethernet-unreliable.2287743/post-31123118
• AX88179 is different to AX88179A. The A at the end indicates macOS support using CDC NCM driver.
• Does not support VLAN with native CDC NCM driver.

Another recommended chipset is Realtek RTL8156B: https://khronokernel.github.io/macos/2021/11/22/PCIE-ETHERNET.html

Proxmox

https://forum.proxmox.com/threads/solved-the-problem-problem-with-2-usb-network-cards-asix-ax88179.101732/

Recommended chipset: RTL8153.

Issue with AX88179: starts not active. Solution is to create a bridge.

https://forum.proxmox.com/threads/ax88179_178a-c-fixed-for-proxmox-e-g-use-with-freebsd-opnsense.60879/

One of my DSM virtual instances had a crashed volume. A crashed volume doesn’t necessarily mean lost data. It crashed for some reason and DSM suggests you to copy your data elsewhere before it becomes worse.

However, my instance is a virtual machine using a network block storage device that has its own protection built-in. My hunch is it crashed because of a network failure and DSM marked it as crashed.

Here’s the process how to fix a crashed volume

Connect to the DSM instance either via SSH or Console (serial).

I use Proxmox and have the option to connect using my virtual serial port.

qm term <VMID>

Once connected, stop all NAS services except for SSH.

sudo syno_poweroff_task -d

Get the raid array information. Look for array that have [E] which means it has an error. Take note of the devices name (e.g. md2 and sdg3).

cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4] [raidF1]
md2 : active raid1 sdg3[0](E) 532048896 blocks super 1.2 [1/1] [E]
md1 : active raid1 sdg2[0] 2097088 blocks [16/1] [U_______________]
md0 : active raid1 sdg1[0] 2490176 blocks [16/1] [U_______________]

To retain the same raid array UUID when it will be recreated later on, we need to get that info. Take note of UUID and Array UUID which should match.

sudo mdadm --detail /dev/md2
/dev/md2:
        Version : 1.2
  Creation Time : Tue Apr 5 11:13:15 2022
     Raid Level : raid1
     Array Size : 532048896 (507.40 GiB 544.82 GB)
  Used Dev Size : 532048896 (507.40 GiB 544.82 GB)
   Raid Devices : 1
  Total Devices : 1
    Persistence : Superblock is persistent

    Update Time : Tue Jan 3 15:17:40 2023
          State : clean, FAILED
 Active Devices : 1
Working Devices : 1
 Failed Devices : 0
  Spare Devices : 0

           Name : JAJA-NVR:2 (local to host JAJA-NVR)
           UUID : 4c38a5c6:7d2b9e1e:76678f10:b7f5e176
         Events : 28

    Number   Major   Minor   RaidDevice State
       0       8       99        0      faulty active sync   /dev/sdg3

sudo mdadm --examine /dev/sdg3
/dev/sdg3:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x0
     Array UUID : 4c38a5c6:7d2b9e1e:76678f10:b7f5e176
           Name : JAJA-NVR:2 (local to host JAJA-NVR)
  Creation Time : Tue Apr 5 11:13:15 2022
     Raid Level : raid1
   Raid Devices : 1
  Avail Dev Size : 1064097792 (507.40 GiB 544.82 GB)
     Array Size : 532048896 (507.40 GiB 544.82 GB)
  Data Offset : 2048 sectors
 Super Offset : 8 sectors
   Unused Space : before=1968 sectors, after=0 sectors
          State : clean
    Device UUID : 40be52e1:68f734ef:980cfaa5:103c5fa6

    Update Time : Tue Jan 3 15:17:40 2023
       Checksum : afa92c2 - correct
         Events : 28

   Device Role : Active device 0
   Array State : A ('A' == active, '.' == missing, 'R' == replacing)

Stop the raid array:

sudo mdadm --stop /dev/md2
[499024.611228] md2: detected capacity change from 544818069504 to 0
[499024.612272] md: md2: set sdg3 to auto_remap [0]
[499024.613155] md: md2 stopped.
[499024.613792] md: unbind<sdg3>
[499024.618114] md: export_rdev(sdg3)
mdadm: stopped /dev/md2

Recreate the raid array:

sudo mdadm --create --force /dev/md2 --level=1 --metadata=1.2 --raid-devices=1 /dev/sdg3 --uuid=4c38a5c6:7d2b9e1e:76678f10:b7f5e176
mdadm: /dev/sdg3 appears to be part of a raid array:
       level=raid1 devices=1 ctime=Tue Jan 3 15:21:44 2023
Continue creating array? y
[499345.180631] md: bind<sdg3>
[499345.182421] md/raid1:md2: active with 1 out of 1 mirrors
[499345.185220] md2: detected capacity change from 0 to 544818069504
mdadm: array /dev/md2 started.
[499345.201216] md2: unknown partition table

Reboot:

sudo reboot

Check the DSM dashboard:

If everything went as expected, you should see the volume as healthy again.

TLDR

# Stop all NAS services except from SSH
sudo syno_poweroff_task -d

# Get crashed volume information (e.g. /dev/md2 and /dev/sdg3)
cat /proc/mdstat

# Get raid array UUID (e.g. 4c38a5c6:7d2b9e1e:76678f10:b7f5e176)
sudo mdadm --detail /dev/md2

# Stop raid array
sudo mdadm --stop /dev/md2

# Re-add volume to the raid array
sudo mdadm --create --force /dev/md2 --level=1 --metadata=1.2 --raid-devices=1 /dev/sdg3 --uuid=4c38a5c6:7d2b9e1e:76678f10:b7f5e176

# Verify it's added without error
cat /proc/mdstat

# Reboot
sudo reboot

Reference: https://xpenology.com/forum/topic/29221-howto-repair-a-clean-volume-who-stays-crashed-volume/?tab=comments#comment-144862

One of the reasons why I don’t publish regularly is I pre-judge what I write if it’s publish-worthy.

At Human Made, we use a WordPress Multisite. Each area/interest of the organization has it’s own site. This fits nicely with my internal structure. I have multiple interest with varying degrees, and only work on those interests only when I feel like working on it.

What I did is convert this personal site to a multisite too and started creating sites for topics I’ve been putting my energy on. This removes the hesitation if it’s worth posting since it will be in it’s own little space. I can be as technical as I need to be. The audience is my future self, and probably my kids if they also happen to stumble on the same interest.

One topic I’ve been spending a lot of time on recently is FTTH. Here’s an example post of installing NAP box for my ODM: https://www.jericoaragon.com/fiber/2022/12/20/installing-my-first-nap-and-two-clients/

My plan is to document my progress using posts and compile elaborate knowledge base using pages.

Grabe. I did not expect that it will take 4 days to add 2x 10TB volume to my NAS.

Started at 2022-12-15 18:27:19

Finished at 2022-12-19 22:06:27

I also realized the importance of UPS. Between adding the volume and completion, there was 20 seconds of electric interruption. That would have risk data corruption kung namatay yung NAS in the middle of the operation.

I’ve upgraded my family’s NAS to add another 2x 10TB drives.

I added the drive 2 days ago. Up until now, it’s still in progress. I did not expect it’ll be this slow. Good thing, I’m not in a rush.

Last week, we had a 4-hour power outage. Since all our networking equipment is plugged in a UPS, I assumed internet will be up for at least an hour. When I check the logs though, it was only around 15 minutes. This meant the UPS battery needs replacement.

Now, I have an obsession (lol) with uptime. When I searched for a replacement, I was also thinking if it’s possible to make UPS run time longer. It turns out that it’s possible by replacing it with a larger capacity battery. But there are safety precautions.

Safety precautions

Don’t use maximum load of the UPS

Replacing the battery with a larger capacity means that the UPS will run longer than it’s designed. There’s a risk of the inverter overheating if it’s running at high load for long periods.

To prevent this, make sure that connected devices is not using more than 80{5fcd3cbc9de14e1587c4b983f08e4837fa7ae8985dc66bae235a2c5aa0d68677} of rated watts. My UPS is 300w. The devices connected are modem, router, PoE switch, and a low-power server for Omada Controller and PiHole. It totals only to 40-50 watts which is only 16{5fcd3cbc9de14e1587c4b983f08e4837fa7ae8985dc66bae235a2c5aa0d68677} of my UPS rated watts.

I did a 2-hour test running on battery. There was no notable heating with the inverter.

Use the same wire gauge to extend connectors

Larger capacity battery also means it’s bigger. It would have to be outside the UPS casing. Using thin wires will introduce a risk of it heating up and might cause fire.

I didn’t understand how to compute what’s appropriate for this, I just used the same wire gauge as the connector inside the UPS (looked like 12 AWG).

Other than that, things should be fine.

The battery included with it was 12v 4.5Ah. I replaced it with 12v 25Ah (5x more than the original battery). This should provide 4-5 hour run time in case of power loss. I then cut a hole in the side of UPS casing to pass through the wire to the battery.

It’s already installed but we haven’t had a power outage yet to actually see how it will actually perform.

When fiber internet was rolled out at my Mom’s home, I had this itch to provide internet for the whole compound. She have an ice cream business and she provides housing to ice cream vendors (sorbetero) together with their families. Around half the compound are sorbeteros. Most of which will benefit if they don’t have to add another expense for internet.

Fiber made bandwidth cheap (P2699 for 100mbps). I knew that a 100mbps plan is more than enough for everyone there. The bottleneck is the device the comes with the internet plan. The device PLDT provides is actually an all-in-one that combines a modem, router, switch, and an access point (WiFi).

To increase coverage, I need to split out the access point and use a separate device.

Project goal

• Share internet that covers the whole compound
• Performance should be adequate for remote learning or work-from-home (Zoom, Google Meet)
• Have the same experience as having their own WiFi
• Roaming-capable (when people move, their device will automatically switch to the next nearest access point)
• Minimal restrictions

Implementation

I opted to go with a business-grade solution. Big factor is a single dashboard to manage all access points. I was initially considering Ubiquity, but then found out about TP-Link Omada which is half the price.

• I get notified through Omada app if the internet is down or if any of the access point stops working
• There are only 2 SSIDs (WiFi name) for 7 access points. One personal and another for the free WiFi. Each has it’s own subnet.
• Free WiFi’s subnet is limited to 50mbps to guarantee that the personal network always have bandwidth available
• Each device is further limited to 20mbps (initially this was 10mbps but utilization rate was low, I bumped it up to 20mbps)
• I had to block Mobile Legends because kids from other compound started coming to our compound to have a tournament until late night. Blocking is just another ACL rule to block port 30000 to 31000. ML stopped loading after this has been applied.

Interesting metrics

• 7 access points covered more than 20 households
• Average internet utilization is only around 20%
• Average traffic is around 180GB download and 15GB upload daily
• Max connected clients so far was 90+ devices
• It’s been running since May 2021. Things has been relatively stable. Downtime was only when there’s no electricity.

Cost

Next step: better cable management

Item	Unit Price	Qty	Total
TP Link Router R509	₱2,180.00	1	₱2,180.00
TP Link 5-port PoE Switch	₱1,440.00	1	₱1,440.00
TP Link EAP110-Outdoor	₱1,400.00	2	₱2,800.00
TP Link EAP225-Outdoor	₱2,900.00	4	₱11,600.00
TP Link EAP235-Wall	₱2,900.00	1	₱2,900.00
Thinkcentre m73p	₱2,500.00	1	₱2,500.00
Omni plugs	₱442.77	1	₱442.77
Waterproof Junction	₱580.23	1	₱580.23
305m CAT6 outdoor cable	₱2,250.00	1	₱2,250.00
Total			₱24,443.00

I could have implement the whole thing 50% cheaper but it’ll be a pain to maintain and less fun to do. Since I’m doing this for free, I might as well enjoy haha.

Why

I see this as a hobby. I’ve always been fascinated with computer networking for as long as I can remember.

The 25k I spent could’ve easily been another gadget where only I would benefit (and add another stuff to my life). Spending it on this instead accomplishes two things: 1) I had fun planning, figuring things out, and setting it up 2) It has good net effect because a lot of people are getting value from it. And that makes me happy.

I also learned a lot. I finally understood how VLAN works. It’s nice that I can map an SSID to a VLAN to have it’s own network. Power-over-Ethernet was also nice because there’s only one cable for power and data (CAT6).

Overall it was worthwhile.

My role at work is partly DevOps. This means that there are rare instances that even if I’m not at home (where internet is reliably available), I might get an alert where a server is not working as it should.

I have mobile data plan from Globe for that. Globe is usually good within cities. The more remote I get though, the less reliable it becomes. My initial solution was to get another phone and line from Smart (competitor).

However, I have minimalist tendency. Paying for two plans monthly feels wasteful. Especially during this pandemic where I barely use both.

I also didn’t like having two phones with me all the time.

Solution: Globe eSim + Smart Magic Sim

My primary network is Globe. I don’t mind paying monthly for this because it’s the same number I used for years. Good thing that they offer eSim which is supported by my phone.

The process was easy. Go to Globe store, ask for conversion from physical sim to eSim. They will then provide a QR code a phone can scan to register the number. That’s it. The eSim solution eliminated my need to carry two phones all the time because I can put in another sim to my phone.

For Smart, I recently found out that they are offering a new product called Magic Sim. It has non-expiring data at P399 for 24GB. Non-expiring! I only need to pay for it when I use it. I dropped the Smart plan I used to have which I was paying for P3500/monthly (easy to justify pre-pandemic) and replaced it with this one.

End result is I have a single phone with two sims from different network. I’m only paying monthly for my primary network. My phone auto-switches network depending on availability of internet.

#

I know this is borderline penny-pinching (which I try not to do). But I like keeping my personal lifestyle low-cost. I also enjoyed eliminating the waste I feel every time I do my budgeting.

https://www.youtube.com/watch?v=j_ksJJ1eVmc

Every time Julie create contracts for her clients, she always complains. It’s one of the things she isn’t looking forward to do in her business. She have this Adobe Illustrator file she manually edits for every client. Even computation itself is manual work. Her process looks like:

1. Look for her laptop
2. Look for laptop’s charger because it’s been a few days before she last used it 😄
3. Open her Adobe Illustrator file
4. Dig in client details. Is it in Facebook Messenger, Viber, email?
5. Add client details
6. Add payment terms
7. Save to Dropbox
8. Send to client for signing

All this takes her around 10-15 minutes per contract. At the end of it sumasakit daw ulo niya. At one point I told her I make her a “contract maker” for her phone. She will only have to put in client details, then it will produce a PDF based on her AI template. This video is our initial MVP (minimum viable product). This removed half the steps of her current process. This also removed the need to have her laptop around to create contracts. Less friction to do as soon as her clients paid the down payment, with all the wedding details still fresh from their conversation. Computation and breakdown of payment terms are done automatically. She tried it on an actual client who followed up her contract. It took her less than 5 minutes to do. Most of all, she doesn’t get headache doing it anymore. 🙂

I have been out of track for a while now. Julie is starting to get worried that I’m getting a little too present-oriented (YOLO lyf). I think the problem starts when I stop checking in life metrics that directly correlate to my “sharpness”. Having no idea of how am I doing makes me care less of my performance. Resistance to know grows because I might not like what I find. Somebody said that if you can’t track it, you can’t improve it.

What do I need to track?

I have 2 key performance indicators (KPI) that are correlated with my work capacity and ability to plan about the future:

1. Screen Time - How much time I spend using a computer. This is tracked by running RescueTime in the background of my computer all the time.
2. Intentional Work Time - How much time I spend intentionally working on something. “Intention” is the keyword. Regardless of how much time I spend on a task, the important thing is if I am aiming at a clear end goal. Opposite of this is jumping from one distraction to the next without accomplishing anything concrete. This is tracked by Toggl.

Normally, I should have more or less 40 hours of screen time per week and around 20 hours of intentional work time. If I dip below 40 hours of screen time, it usually means I’m trying to avoid working by doing something else. When I’m at my sharpest, I am more self-aware, this makes most of what I do to have clear intentions before I do them. The closer intentional work time to screen time, the better.

Out of sight, out of mind. The opposite is true too.

I need those 2 KPIs to be always in my sight. This will give me a general grasp of how I’m doing based on a concrete data. I can do my “getting back on track” checklist if I’m doing poorly. I have the skillset to quickly whip up a simple dashboard that displays those data. I also have a spare tablet that I’m not using. I can use it to display my KPIs on 24/7 with very little power consumption (5V/1A).

Getting my hands dirty

1. Preps - I did not bother using any framework. It doesn’t matter how dirty the setup and the coding is since it’s personal use only. The goal is to display the correct data. Originally I planned to use curl to fetch the data but it looks like using API wrappers will be much faster and future-proof. I always went to the fastest solution because I want to have a working dashboard in one sitting. I started 2 files: index.php for gluing all the parts and composer.json for dependency management.
2. Getting RescueTime data - For RescueTime, I used borivojevic/rescuetime.

$rescuetime_key = "RESCUE_TIME_API_KEY";

use RescueTime\RequestQueryParameters as Params;
use RescueTime\Client;

$client = new Client($rescuetime_key);
$day = date('w');
$week_start = date('Y-m-d', strtotime('-'.$day.' days'));
$week_end = date('Y-m-d', strtotime('+'.(6-$day).' days'));

// Fetch activities for this week
$activities = $client->getActivities(
  new Params([
    'perspective' => 'interval',
    'resolution_time' => 'week',
    'restrict_begin' => new \DateTime($week_start),
    'restrict_end' => new \DateTime($week_end)
  ])
);

// Compute total time spent
$totalTimeSpent = 0;
foreach ($activities as $a) {
  $totalTimeSpent += $a->getTimeSpentSeconds();
}

$totalScreenTimeThisWeek = number_format(round($totalTimeSpent/60/60, 2), 2);

3. Getting Toggl data - For Toggl, I used ixudra/toggl.

$workspaceId = 0;
$apiToken = 'TOGGL_API_KEY';

$togglService = new \Ixudra\Toggl\TogglService($workspaceId, $apiToken);
$response = $togglService->summaryThisWeek();
$totalIntentionalWorkThisWeek = number_format(
  round($response->total_grand/1000/60/60, 2),
  2
);

4. Displaying computed data to the page

<div>
  <p>Screen Time</p>
  <h1><?php echo $totalScreenTimeThisWeek; ?></h1>
</div>
<div>
  <p>Intentional Work</p>
  <h1><?php echo $totalIntentionalWorkThisWeek; ?></h1>
</div>

5. Keeping the data up-to-date

<meta http-equiv="refresh" content="600">

Putting everything together

I ended up having a working dashboard in around 1.5h of intentional work. After the coding is done, I uploaded it to my server and loaded it in the tablet.

It sits on a shelf where I can see it everyday (together with The Daily Stoic, which I read 1 page everyday).

This sets a framework that I can update as needed if ever I need a new KPI to track. The process of adding a new data source will be just 1) figuring out how to get the data 2) displaying it. The boilerplate is done. The friction to update will be minimal. If you are interested in this project, have a KPI to recommend, or just want to talk in general — feel free to reach out!

Backup to Amazon S3 using WHM is natively supported. However, I ran into trouble that the AWS keys I setup isn’t working. It turns out that my bucket has to have no period in its name otherwise it will fail because of an SSL error. To configure automatic backup from WHM to Amazon S3:

Create an S3 bucket and get AWS credentials

1. Go to AWS S3 Console
2. Create a new bucket - make sure the bucket name has no period in its name
3. Click your account name on the top right to show a dropdown menu
4. Select My Security Credentials
5. Click Access Keys
6. Click Create New Access Key
7. Keep this window open because we need to get the keys later

Setting up Amazon S3 as a new backup destination

1. Login in WHM admin page
2. Click Backup Configuration in the sidebar
3. In Additional Destinations, select Amazon S3
4. Click Create new destination
5. Fill-in destination name - the name you want for this destination e.g. Jerico’s S3
6. Fill-in the bucket name you created earlier
7. Paste Access Key ID and Secret Access Key from AWS S3 Console
8. Set Timeout to 300
9. Click Save and Validate Destination

If everything goes well, you now have an S3 destination for your backups. You can start configuring how and when you want your backups to run. To start the backup, you need to SSH in to your server and run /usr/local/cpanel/bin/backup