Nduli's World

GUIX OS with GeneNetwork

2024-07-22T00:00:00+03:00

Guix OS install involves downloading the iso; setting it on a flash disk with: dd if=guix.iso of=/dev/sdX status=progress; and using the graphical guide. Some quirks I got were:

I needed an internet connection but wifi didn't work out of the box. I used USB tethering though.
I forgot to edit the final configuration to UEFI. To boot, I used grub by:

# ref: https://superuser.com/a/1512531
set pager=1
ls
# find the drive that has guix_os by searching for /etc/config.scm file
cat drive_from_ls/etc/config.scm # e.g. cat (hd0,1)/etc/config.scm
set root=drive_from_ls # e.g. set root=(hd0,1)
cat /boot/grub/grub.cfg
# run the commands in the menuentry manually
linux /gnu/store/......
initrd /gnu/store/.....
# boot the OS
boot

I changed my bootloader config in /etc/config.scm to fix the efi issue

;; Ref: https://guix.gnu.org/manual/en/html_node/Bootloader-Configuration.html#index-bootloader_002dconfiguration
(bootloader (bootloader-configuration
  (bootloader grub-efi-bootloader)
  (targets '("/boot/efi"))
  (keyboard-layout keyboard-layout)))

and rebuild the OS with sudo guix system reconfigure --allow-downgrades /etc/config.scm - I experienced screen flickering in my terminal and modified my boot parameters to fix it:

;; ref: https://wiki.archlinux.org/title/Intel_graphics#Screen_flickering
(bootloader (bootloader-configuration
              (bootloader grub-efi-bootloader)
              (targets '("/boot/efi"))
              (keyboard-layout keyboard-layout)))
(kernel-arguments (list "i915.enable_psr=0"))

Quality of Life Changes

Zsh as default shell

Change /etc/config.scm to have:

;; ref: https://lists.gnu.org/archive/html/help-guix/2017-07/msg00024.html
;; added this
(use-package-modules shells)

(users (cons* (user-account
                (name "rookie")
                (comment "rookie")
                (group "users")
                (home-directory "/home/rookie")
                ;; added this line
                (shell #~(string-append #$zsh "/bin/zsh))
                (supplementary-groups '("wheel" "netdev" "audio" "vidoe")))
            %base-user-accounts))

When installing oh-my-zsh I also got an error: ZSH not installed and I had to do an extra guix install zsh.

Non Guix

Non Guix Reference

For wifi to work, I modified /etc/config.scm to have:

;; non free linux module
(use-modules (nongnu packages linux)
             (nongnu system linux-initrd))

(operating-system
  (kernel linux)
  (firmware (list linux-firmware))

The README did most of the heavy lifting, but I couldn't figure out how to add the substitutes.

;; what ended up in my services section
  (services
   (append (list (service xfce-desktop-service-type)

                 ;; To configure OpenSSH, pass an 'openssh-configuration'
                 ;; record as a second argument to 'service' below.
                 (service openssh-service-type)
                 (service cups-service-type)
                 (set-xorg-configuration
                  (xorg-configuration (keyboard-layout keyboard-layout))))

           ;; This is the default list of services we
           ;; are appending to.
           ;; %desktop-services)
                (modify-services %desktop-services
                     (guix-service-type config => (guix-configuration
                       (inherit config)
                       (substitute-urls
                        (append (list "https://substitutes.nonguix.org")
                          %default-substitute-urls))
                       (authorized-keys
                        (append (list (plain-file "non-guix.pub"
                                                ;; output from https://substitutes.nonguix.org/signing-key.pub
                                                ;; merged into one line
                                                  "(public-key (ecc (curve Ed25519) (q #C1FD53E5D4CE971933EC50C9F307AE2171A2D3B52C804642A7A35F84F3A4EA98#)))"
                                                  ))

                          %default-authorized-guix-keys))))))
   )

Locale Bug

I couldn't get tmux to start, getting tmux: invalid LC_ALL, LC_CTYPE or LANG and running locale -a failed too. The root cause was that my applications were built on a different version of glibc and running guix upgrade fixed this.

GeneNetwork Setup

Follow the instructions here to set up genenetwork

Small changes:

# pick the channels file from 
curl https://ci.genenetwork.org/channels.scm > channels.scm
# gn2 setup
guix pull -C channels.scm -p ~/.guix-extra-profiles/gn2
GUIX_PROFILE=$HOME/.guix-extra-profiles/gn2
. $GUIX_PROFILE/etc/profile
guix install genenetwork2 -p ~/.guix-extra-profiles/gn2
guix install genenetwork3 -p ~/.guix-extra-profiles/gn2 # setups an older version

# gn3 setup
guix pull -C channels.scm -p ~/.guix-extra-profiles/gn3
GUIX_PROFILE=$HOME/.guix-extra-profiles/gn3
. $GUIX_PROFILE/etc/profile
guix install genenetwork3 -p ~/.guix-extra-profiles/gn3
guix install genenetwork2 -p ~/.guix-extra-profiles/gn3
guix package -i python-mysqlclient -p ~/.guix-extra-profiles/gn3
guix package -i python-pyld -p ~/.guix-extra-profiles/gn3
pytest -k unit_tests # succeeds after this
pytest # still fails
FLASK_DEBUG=1 FLASK_APP="main.py" flask run --port=8080 # works

# gn-auth set up is a bit different from the above
# the README is clear in this case
# just make sure you have genenetwork2 and genetwork3 installed in the profile

guix pull -C channels.scm -p ~/.guix-extra-profiles/gn-auth
GUIX_PROFILE=$HOME/.guix-extra-profiles/gn-auth
. $GUIX_PROFILE/etc/profile
guix install genenetwork3 -p ~/.guix-extra-profiles/gn-auth
guix install genenetwork2 -p ~/.guix-extra-profiles/gn-auth
guix package --install-from-file=guix.scm -p ~/.guix-extra-profiles/gn-auth

Generate HTML when Using Markdown in Vimwiki

2023-09-02T00:00:00+03:00

I moved to markdown in vimwiki but couldn't generate html. VimwikiAll2HTML only supported vimwiki syntax. I explored static site generators looking for one that:

didn't require moving my files and folders.
didn't need a lot of configuration to set up.

I tested out mkdocs, zola, docusaurus and vitepress and only vitepress met both conditions. It also worked out the box and I liked the default theme.

npm install -D vitepress
npx vitepress init

Note:

This works with "vitepress": "^1.0.0-beta.3" in my package.json.
This overrides index.md, so have a back up. I used git to restore it.

I generated the hmtl using:

npx run docs:build

The sidebars didn't look great though, and I used vitepress-plugin-auto-sidebar to set this to by my folders. It's config looks like:

sidebar: getSidebar({ contentRoot: '/', contentDirs: ['team'], collapsible: true, collapsed: true })

and this forced me to set up a long list of contentDirs. I instead used an ignore list and generated my contentDirs from code with:

const EXCLUDE_DIRS: string[] = ['node_modules', 'diary', 'scripts'];
const CONTENT_ROOT = '/';

function getSideBarFolders(): string[] {
  var content_root = path.join(process.cwd(), CONTENT_ROOT)
  return fs.readdirSync(content_root, {withFileTypes: true})
           .filter(f => f.isDirectory() && !f.name.startsWith('.') && !EXCLUDE_DIRS.includes(f.name))
           .map(f => f.name)
           .sort();
}

changing the plugin's config to:

sidebar: getSidebar({ contentRoot: CONTENT_ROOT, contentDirs: getSideBarFolders(), collapsible: true, collapsed: true }),

I added a post-commit hook that would update my vimwiki site any time I changed my wiki:

set -euo pipefail

cd "$(git rev-parse --show-toplevel)"
npm run docs:build
rsync --recursive --archive --update --verbose --compress -P .vitepress/dist/ user@ip:/path/to/folder

Migrate From Vimwiki To Markdown Syntax

2023-07-29T00:00:00+03:00

Vimwiki supports markdown and I switched to this because:

I didn't use the default syntax anywhere else.
I could convert to html using only VimwikiAll2HTML.

Fixing My Config

Vimwiki expects some options set before it loads (ref). With lazy, I use init like:

{ 'vimwiki/vimwiki',
  branch = 'dev',
  dependencies = {'mattn/calendar-vim'},
  init = function()
    vim.g.vimwiki_list = {
      {path = "~/vimwiki", path_html= "~/vimwiki/public_html", auto_tags = 1, auto_diary_index = 1, syntax = 'markdown', ext = '.md'},
      -- experimental folder for markdown conversion
      {path = "~/projects/vimwiki", path_html= "~/projects/vimwiki_markdown/public_html", syntax = 'markdown', ext = '.md'}, }
    vim.g.vimwiki_markdown_link_ext = 1
    vim.g.vimwiki_stripsym = ' '
    vim.g.vimwiki_global_ext = 0
  end
},

Pandoc Conversion

I tested pandoc for conversion with the file:

== Title ==

- [o] list item i:
  - [.] sublist
    - [X] sub sub
    - [ ] sub usb
    - [ ] sub sub1
  - [X] final

TODO: task 1

%% a comment I've left to address later

I'm a normal paragraph.

[[link to file]]

{{{bash
  ls -lah
  find . -iname '*.wiki'
}}}

Another code block

{{{
  ls -lah
  find . -iname '*.wiki'
}}}

Running pandoc --from vimwiki --to commonmark_x trial.wiki gave me:

╰─$ pandoc --from vimwiki --to commonmark_x trial.wiki
## Title {#Title}

- []{.done2}list item i:
  - []{.done1}sublist
    - []{.done4}sub sub
    - []{.done0}sub usb
    - []{.done0}sub sub1
  - []{.done4}final

[TODO:]{.todo} task 1

I\'m a normal paragraph.

[link to file](link to file "wikilink")

``` bash
  ls -lah
  find . -iname '*.wiki'
```

Another code block

      ls -lah
      find . -iname '*.wiki'

I had these problems with the conversion:

the checklist generated wouldn't work with vimwiki
I didn't want the escaped ' character in my files
The link to a file had the string "wikilink" which wouldn't work in vimwiki
I lost my comment
The last code block wasn't wrapped around ` characters

I'd combine sed with pandoc to get what I wanted.

Preprocessing

I replaced the comment identifier %% with TODO: comment to the prevent losing them during conversion. I also replaced code blocks without a language i.e. {{{ to have bash i.e. {{{bash and I'd replace this when editting later.

╰─$ echo -e "%% save me\nNormal line %% with ending\n{{{" \
      | sed -r -e 's/\{\{\{$/\{\{\{bash/g' \
               -e 's/%%/TODO: comment/g'
TODO: comment save me
Normal line TODO: comment with ending
{{{bash

Post processing

I changed the vimwikie-converted linked i.e. [link_to_file](link_to_file "wikilink") to use the file name i.e. [link_to_file](link_to_file.md).

echo -e "[link_to_file](link_to_file \"wikilink\")" \
  | sed -r -e 's/(\[.*\])\(([^#]*)((.*) "wikilink")\)/\1\(\2.md\4\)/g'

I converted all checklists to github's checklist syntax with:

echo -e "- []{.done4}done\n- []{.done0} not_done" \
  | sed -r -e "s/\[\]\{\.done[0-3]\}/\[ \] /g" \
           -e "s/\[\]\{\.done4\}/\[X\] /g"

and also unquoted all ' characters with:

echo -e "\\'" | sed -e "s/\\\'/\'/g"

Here's the final post processing step:

  sed -r -i -e 's/(\[.*\])\(([^#]*)((.*) "wikilink")\)/\1\(\2.md\4\)/g' \
    -e "s/\\\'/\'/g" \
    -e "s/\[\]\{\.done[0-3]\}/\[ \] /g" \
    -e "s/\[\]\{\.done4\}/\[X\] /g" \
    "$md_file"

Putting It All Together

#!/bin/bash

set -euo pipefail

# files to change extension from to md format
readarray -d '' mv_files < <( \
  find . \( -iwholename '*diary/*.wiki' \
    -or -iwholename '*unorganized_things/*.wiki' \
    -or -iwholename '*design_docs/*.wiki' \
    -and -not -iname 'diary.wiki' \
    -and -not -iname 'index.wiki' \) \
    -print0 
)

for file in "${mv_files[@]}"; do
  md_file="${file%%.wiki}.md"
  mv "$file" "$md_file"
done

readarray -d '' files < <(find . -name "*.wiki" -print0)
for file in "${files[@]}"; do
  md_file="${file%%.wiki}.md"
  sed -r -e 's/\{\{\{$/\{\{\{bash/g' -e 's/%%/TODO: comment/g' "$file" | pandoc --from vimwiki --to commonmark_x -o "$md_file"
  sed -r -i -e 's/(\[.*\])\(([^#]*)((.*) "wikilink")\)/\1\(\2.md\4\)/g' \
    -e "s/\\\'/\'/g" \
    -e "s/\[\]\{\.done[0-3]\}/\[ \] /g" \
    -e "s/\[\]\{\.done4\}/\[X\] /g" \
    "$md_file"
  rm "$file"
done

SC-IM Color Configuration

2023-07-01T00:00:00+03:00

I couldn't read cells in sc-im because there was little contrast between the background and foreground colors. I temporarily fixed this by changing my terminal's color scheme, but I'd prefer to keep it. A permanent solution was to change sc-im's colors, for example:

color "type=INPUT fg=WHITE bg=BLACK"

and changed the colors of the help documentation with:

color "type=NORMAL fg=WHITE bg=BLACK"

The help menu provides more details for the colors to use. You type :help and search for color with \color, getting the explanations for all the types supported. I experimented with a couple of options and saved my final colors in ~/.scimrc. Here's what I had:

color "type=INPUT fg=WHITE bg=DEFAULT_COLOR"
color "type=NORMAL fg=WHITE bg=DEFAULT_COLOR"
color "type=ERROR_MSG fg=RED bg=BLACK"
color "type=CELL_SELECTION fg=BLACK bg=WHITE"
color "type=HEADINGS fg=WHITE bg=DEFAULT_COLOR"
color "type=WELCOME fg=WHITE bg=DEFAULT_COLOR"
color "type=MODE fg=WHITE bg=DEFAULT_COLOR"

I also found a ticket on the problem and presented my solution here.

Dev Experience For Personal Projects

2023-04-29T00:00:00+03:00

My personal projects have poor developer experience. Any time I want to fix a bug or improve a service, I experience a lot of frustration.

For example, I wanted to upgrade the UI of my pomodoro script to have colors for completed and cancelled tasks, better prompts and mark the task status. I couldn't get how the original script worked and any change I made broke something. I added the features but the experience was painful. I didn't have another way to tell that I broke something except from running the script end to end.

I set up my first server manually, without documentation. There were a lot of experiments I did, which means it's impossible to create a similar setup in future or upgrade the same server. A better way would have been to document the set up, or put everything in code using ansible, with comments that explain non-obvious configurations.

I'll try and have at least these in my personal projects to help with dev experience:

easy set up of project
easy and fast testing
clear documentation on manual and non-obvious steps
documentation to run and build the project
documentation on how I deployed the project
explanation for non-obvious debugging steps

Testing GUIX

2023-03-31T00:00:00+03:00

I installed guix with:

# Install nscd (name service cache daemon) to help pick up guix libs
sudo apt install nscd
sudo systemctl enable nscd

# install guix
# Ref: https://guix.gnu.org/manual/en/html_node/Binary-Installation.html
cd /tmp
wget https://git.savannah.gnu.org/cgit/guix.git/plain/etc/guix-install.sh
chmod +x guix-install.sh
sudo ./guix-install.sh

# update guix
guix pull

# log out and in again
# confirm guix-daemon is running with:
ps aux | grep -i guix

#  installing guix locales, used by libc installed via guix
guix install glibc-locales

# ensures guix libs and apps can do https
guix install nss-certs

# Add the following to .profile
GUIX_PROFILE="$HOME/.guix-profile"
GUIX_LOCPATH=$GUIX_PROFILE/lib/locale
. "$GUIX_PROFILE/etc/profile"

I like i3 window manager and the ubuntu provided version doesn't yet support gaps. I installed i3 using the ubuntu package manager (apt), which configured the login manager for i3 in /usr/share/xsessions/. I then installed i3-gaps using guix and it had preference to the i3 installed via apt.

sudo apt-install i3
guix install i3-gaps

I set up fonts using:

guix install fontconfig
fc-cache -rv

and installed other useful packages:

# installing my editor
guix install python neovim python-pynvim git

I update my guix system using:

guix pull
guix package -u

I needed to install neovim that was build with a later version of the gcc-tool chain. I solved this with:

guix package -i neovim --with-c-toolchain=tree-sitter=gcc-tool-chain@12.2

Stack Static Missing Library With Guix

2023-02-25T00:00:00+03:00

I installed stack_static by downloading the binary from github and moving it to .local/bin. I was able to compile haskell programs, but running them failed with the error:

$ stack run
/home/rookie/projects/random/helloworld/.stack-work/install/x86_64-linux-tinfo6/48795ad824c5ab2c527b42cc08a43bf5cd587c042c536072710c2302b997bb41/9.2.5/bin/helloworld-exe: error while loading shared libraries: libgmp.so.10: cannot open shared object file: No such file or directory

ldd showed that the library was missing:

$ ldd .stack-work/install/x86_64-linux-tinfo6/48795ad824c5ab2c527b42cc08a43bf5cd587c042c536072710c2302b997bb41/9.2.5/bin/helloworld-exe
        linux-vdso.so.1 (0x00007ffd0958e000)
        libgmp.so.10 => not found
        libc.so.6 => /gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib/libc.so.6 (0x00007f5b9d530000)
        libm.so.6 => /gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib/libm.so.6 (0x00007f5b9d3ef000)
        librt.so.1 => /gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib/librt.so.1 (0x00007f5b9d3e5000)
        libdl.so.2 => /gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib/libdl.so.2 (0x00007f5b9d3e0000)
        libpthread.so.0 => /gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib/libpthread.so.0 (0x00007f5b9d3be000)
        libgcc_s.so.1 => /gnu/store/4zvswpr2h3b7dvqpvjcdam8vfhyjrmgl-gcc-12.2.0-lib/lib/libgcc_s.so.1 (0x00007f5b9d39d000)
        /gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib/ld-linux-x86-64.so.2 => /gnu/store/ayc9r7162rphy4zjw8ch01pmyh214h82-glibc-2.33/lib/ld-linux-x86-64.so.2 (0x00007f5b9d6f4000)

I noticed the search path for most libraries were guix related, so I ran the OS provided ldd and it found the libgmp.so.10 library.

$ /usr/bin/ldd .stack-work/install/x86_64-linux-tinfo6/48795ad824c5ab2c527b42cc08a43bf5cd587c042c536072710c2302b997bb41/9.2.5/bin/helloworld-exe
        linux-vdso.so.1 (0x00007ffc41cc0000)
        libgmp.so.10 => /lib/x86_64-linux-gnu/libgmp.so.10 (0x00007f4148ad2000)
        libc.so.6 => /gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib/libc.so.6 (0x00007f4148910000)
        libm.so.6 => /gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib/libm.so.6 (0x00007f41487cf000)
        librt.so.1 => /gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib/librt.so.1 (0x00007f41487c5000)
        libdl.so.2 => /gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib/libdl.so.2 (0x00007f41487c0000)
        libpthread.so.0 => /gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib/libpthread.so.0 (0x00007f414879e000)
        libgcc_s.so.1 => /gnu/store/4zvswpr2h3b7dvqpvjcdam8vfhyjrmgl-gcc-12.2.0-lib/lib/libgcc_s.so.1 (0x00007f414877d000)
        /gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib/ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2 (0x00007f4148b68000)

I didn't know how to add the library to guix (guix install gmp didn't fix this). The solution was to compile the program using native OS level utilities.

# opens bash without guix loaded utilities
env -i /bin/bash --login --noprofile --norc  
export PATH=$PATH:~/.local/bin
stack build

The binary worked without an errors.

Comic Site Migration to SQLite

2023-01-07T00:00:00+03:00

Postgres was the database for my comic site but I chose to migrate to sqlite because it would:

be easier to back up
use less resources on my server
be one less service to manage

I also don't get a lot of traffic so it wouldn't have an impact.

The process followed this general pattern:

Change the code to use sqlite instead of postgres. I'd maintain the same paths to the static resources so that I wouldn't have to move these (I stored static resources on the host).
Change the docker-compose yml file to:
- Use a different local port for the sqlite instance
- Use a different project name
Create a json dump from the production server using: python manage.py dumpdata --natural-foreign --natural-primary > /var/backups/comic_server/natural_dump.json
Change ansible to deploy the project to a different folder and run it.

Perform a migration and load the data with:

python manage.py migrate
python manage.py loaddata /var/backups/comic_server/natural_dump.json

Verify the new instance works.
Change nginx configuration to point to the new instance.
Turn down the old instance with docker-compose down. I'll clear out the volumes once I'm sure everything is ok.

Note: python commands run in the docker-compose container by first running: docker compose exec comic_server /bin/bash

Handling Noise in My Head

2022-11-30T00:00:00+03:00

I complain and anger myself a lot. I'll have an internal dialogue that loops over the same points, making me feel worse and worse. I'll sometimes write down why I'm so angry or anxious, and find out my thoughts aren't as important as I wanted to believe.

This is Chatter, where our brain gets stuck in negative thought loops, usually about a past event e.g. something someone told me (rumination) or about the future e.g. what will happen (worry).

I subconsciously like this feeling and sometimes catch myself looking for something to complain or be angry with. I'll look for someone to validate my feelings, having a one sided conversations about my problems and complaints. I won't notice solutions they suggest since I'm too fixated on my problems. Later on, when I reflect on what they said is when I'll see their solution and angle.

What seems to work for me is:

cycling. When I reach some point of exhaustion, I stop thinking and my mind calms down for the rest of the ride.
writing. This makes the fuzzy thoughts less so, and I see what the problem is, rather than feel them.
smiling: Forcing myself to smile makes me look at my thoughts from a different angle, and I've found myself laughing at some of the thoughts I had.

Other ways I've read that might work are:

distanced self talk, where instead of using I/me while thinking I use my name or other pronouns like you, he, she.
practice meditation and mindfulness
do something else that takes my focus from the rambling thoughts like color naming, count, recite, run, tidying up, taking a walk.
Talking/Writing. Thoughts tend to be abstract and unordered and verbalizing these helps generate meaning from the thoughts.
Practice changing my perception:
- Reframe thoughts (cognitive defusion), for example, "life is meaningless" becomes "I'm having a thought that life is meaningless"
- Train myself to recognize a negative thought I fall on, and consciously create a new thought that's based on reality to replace it (Cognitive Behaviour Therapy).
- Instead of trying to change my internal state, accept it with kindness. Change internal dialogue to 'Its hard to feel this way but I can get through this' instead of blaming myself for anxiety.

References

How To Code Review

2022-10-29T00:00:00+03:00

Code reviews improve the health of a system while spreading ownership of code and knowledge. A good review process is fast and we can handle low hanging fruits by automating trivial checks (e.g. running unit tests, style choices) and choosing style guidelines. We can have high level discussions on the changes before coding to prevent rewrites.

Once the changes are ready, the reviewer:

responds quickly to the review request (if not in the middle of deep work). At most 1 day shouldn't pass before they respond.
checks the code and tests for:
- correctness, test coverage, functionality and best practices.
- hard to understand code, unclear names, commented out code, untested code,
- unhandled edge cases, duplication, bad test structure
- too many changes (if these are a lot, ask dev to split the PR).
- comments. They should explain why not what the code is doing (comments != documentation).
- consistency
- documentation updates.
is kind when commenting on code. They:
- comment on what they like in the PR (positive feedback)
- comment on code and not the coder e.g. instead of you're creating many db connections here have this code creates many db connections.
- Use questions instead of statements. Questions feel less like criticism and can trigger creative solutions while also assuming the author knows more than you do e.g:
  - I find this confusing becomes Could you elaborate on how this works?
  - This is unmaintainable becomes What was your vision on maintaining this?
- Use I-messages instead of you-messages i.e. I suggest/think/believe/would, It's hard for me, For me it seems e.g. you are writing cryptic code becomes It's hard for me to grasp what's going on in this code. We see You-messages absolute statements or attacks, and get defensive.
- Use we instead of I/me/you which makes comments less judgemental e.g. could we change this function to do it this way?
ensures the dev implements explanations in code (i.e. rewrite or a comment). Explanations in the review tool won't help future devs.
accepts that there are different solutions and distinguish between common best practices and personal taste. They are okay with compromises.
Use the "Yes, and ... " technique to keep an innovative atmosphere. Don't dismiss fresh and fragile ideas in a draft PR.
Have discussions in public, and ensure code documents these.
is explicit about the action they want. Prefix your comments with emojis or the intent e.g. nit, FYI.
Reach out to dev if they have too many comments or comments that are too long, solving misunderstanding early. Talk these out instead of using comments.
Have a bias for action e.g. approve a PR with minor comments.

References

Asking For Help

2022-08-27T00:00:00+03:00

I get and send bad questions. To avoid this, I write my questions in my editor, review and then copy them to the platform e.g. slack, email. My questions are better, which makes me wonder if there are other changes that could further improve my questions.

A lot of great resources exist online, and here's a checklist I've made for my questions.

Before I consider asking a question, ask:

What do I want? Avoid thinking that the solution in mind is the problem. I can tack on the failed solution to the question if need be. This prevents the XY problem when I'm making the question.
Am I forcing my experience, knowledge or the technology I know to the problem (i.e. einstellung?
Spend time making the question before reaching out to someone. This prevents sending a hello with no context, since I have the question to add after the greeting.
Do my research, and note down all the findings I've got. This helps when adding details to the question.

For example, I built an image on a server, but failed severally because the server ran out of memory. I spent a lot of time trying to figure out how to limit docker memory, but this wasn't the goal. Following the above process could have helped.

What did I want? I wanted a custom image on the server.
Am I forcing the technology I know or the solution I've thought out? Yes I was. The server ran out of memory severally and I tried to find ways to limit the memory while building the image. I didn't even stop to consider that there would be another option to get this image in.
Spend time making the question before reaching out to someone. At this point I realized that I was solving the wrong problem, and decided to find other options for what I wanted.
Do my research and note down all findings. While researching the original problem, I found that I can load up a locally build image to a server by using docker load, which solve the problem, hence no need to ask.

Once I've done the above and the problem persists, I make and refine the question by:

Having meaningful subject headers.
Starting with the goal, then the steps I've taken, hence I ensure others will know if my steps were wrong or if there were other options I didn't consider.
Editing the question to be short but informative. I can include:
- the environment this occurred (if relevant)
- the symptoms in chronological order
- the research done
- a way to reproduce the error (if possible).
- screenshots if its a UI bug
Trimming down code snippets and test cases to the smallest bug-demonstrating version.
Avoiding open-ended questions by being explicit in what I want. The less effort my request needs the likelier I'll get a response e.g. 'Can someone explain X?' forces someone to create a detailed explanation while 'Can someone provide a pointer to a good explanation of X' means they only send you to a relevant blog post.
Writing in clear, grammatical and well spelled language.
Avoiding yes or no questions.

Once you have the question, then:

Look for a relevant public channel. Avoid private ones or personal messages since public ones ensure more people see the question and learn from it.
Follow up with a brief note on the solution to the thread. Upvote helpful answers and accept any that solved the problem.
Don't ask questions in ongoing threads since those not following the thread will not see it.
Be courteous, use please and 'thanks for your attention', etc.

References

My Firefox Configuration

2022-07-02T00:00:00+03:00

I like firefox, and I've tweaked it to be ideal for me. The extensions I have are:

Dark reader: I prefer dark themes, so this applies one to websites. This works great except for my bank's website, which I've excluded in the plugin.

Tridactyl: provides vim keybindings. It's intuitive and I'm fond of how it implemented bookmarks. I changed the theme to shydactyl-dark using:

colors --url https://raw.githubusercontent.com/eeshugerman/shydactyl-variations/master/dist/shydactyl-dark.css shydactyl-dark
colourscheme shydactyl-dark

When opening a new tab, there'd be a white canvas that immediately switched to dark-mode, a flash bang, which was disorienting. I found a solution on this github comment and applied it by opening $HOME/.mozilla/firefox/profile-dir/chrome/userContent.css and adding:

/*
   dark background in new tabs without a white flash (with tridactyl newtab)
   @see: https://github.com/tridactyl/tridactyl/issues/2510
*/
:root {
  --tridactyl-bg: #1d1b19 !important;
  --tridactyl-fg: white !important;
}

/*
    set the background color of the new tab page (without tridactyl or with tridactyl without newtab)
*/
@-moz-document url-prefix(about:home), url-prefix(about:newtab) {
  body {
    background: #1d1b19;
  }
}

uBlock origin: ad blocker
Leechblock NG: blocks sites that I don't want to visit. I have to add these sites to a block list though, but once done, it's difficult to open them. I've added youtube, twitter and more, and this helps me concentrate on important tasks.
Remove youtube suggestions: blocks distracting features on youtube. I've removed the Explore link, Shorts link and End of Video suggestions.
Tab Counter: shows how many windows and tabs I have open, and the number of tabs in the current window. This gives me a rough idea of whether my tabs are becoming unmanageable.
SponsorBlock: skips sponsor sections in youtube videos.

I also change the following configs:

dom.event.clipboardevents.enabled = False
layout.css.devPixelsPerPx = 1.25
ui.systemUsesDarkTheme = true
toolkit.legacyUserProfileCustomizations.stylesheets = true
browser.display.background_color = #000000
browser.display.foreground_color = #ffffff

I previously had a problem where the first menu item on right click would be chosen in firefox. I haven't experienced this in a while, but here's a solution just in case: soln for mouse clicking

My Opinions on expenses

2022-06-04T00:00:00+03:00

Cheap items

I read this article about buying the cheapest tool first, and if I use it until it breaks, then I can buy the most expensive one. I've tried to apply this to my purchases, and I've realized that I am sometimes okay with the cheap things I've bought

For example, I wanted a laptop, and saved up Ksh 150K for it, but chose to try out an older thinkpad model first. I bought a refurbished thinkpad t440 for Ksh 30K and I love it. It has most of the things I need and great linux support. The only gotcha is playing AAA games, but this led me to trying out niche games and I've found some hidden gems. If this laptop ever breaks, I'll get a similar one. I don't see the point of spending 5 times the amount on something that is marginally better (for my needs that is).

Sometimes the most expensive item is difficult to maintain, especially when parts need to be replaced. When we moved into our home, we bought the most expensive shower head our electrician recommended, and it worked for a year before the heating element broke. The electrician tried to find a replacement, but couldn't, so we bought a new shower head. But this time we got the lower priced ones that are more common. This also happened with my brother's laptop (an ultrabook), where he'd accidentally poured water on the keyboard, and it needed to be replaced. It took weeks to find a replacement, and it was costly.

I now purchase cheap items mostly because:

It provides a chance to see if I need it or not.
The cheap item are sometimes capable of what I need.
The cheap items are sometimes easier to repair.

I avoid cheap items that have too many features for the price though e.g. a convertible laptop, with a touch screen, support for a digital pen, light weight and more. At that price point, these features don't make sense and there are some shortcuts made e.g. poor build quality.

Impulse purchases

I sometimes get an urge to buy something I've seen, because seeing causes my brain to find uses for it. For example, I'll see a camera and start imagining all the cool photos I can take, while ignoring that I rarely take pictures on my smart phone. Being aware of this helps a bit, especially when it comes to places I avoid (ref: atomic habits).

Supermarkets and malls may be designed with this in mind. In malls, I easily get lost when looking for a particular shop, meaning that I see a lot more things that triggers the impulse. I try to avoid malls and look for alternatives. When shopping, I make a list and stick to it, avoiding cues from the supermarket.

I use leetblock and adblockers on my computers to block websites that provide these cues. This makes it really hard to visit them.

Experiences vs Things

I also liked this article on buying things not experiences, and learnt that there's a blurry boundary between items and experiences, in that there are experience like things e.g. owning a workshop, and there are thing-like experiences e.g. an instagrammable vacation. Experiences are preferred because of limits in our lives e.g. we don't have enough space for them, or it could be a way to show that we're better in a world of material abundance, hence the 'buy experiences not things' mantra.

This ignores that we can get things that can provide multiple life changing experiences over the course of their ownership, so long as they're carefully chosen. A naive example would be to purchase a gaming rig rather than making payments at a gaming shop. This provides multiple chances to explore games you enjoy, playing with friends and even meeting random people online. Another similar example provided is owning a large living room, which provides a change to host friends and expand your network.

So where possible, consider the experiences of making a purchase, rather than just brushing it off as materialism. This though means I should also be aware of the biases mentioned in the previous section, and I try to balance both of these.

Moving home directory to another partition

2022-04-30T00:00:00+03:00

I'd need to set up ubuntu, but I didn't want to lose my home directory. Everything was in one partition, so I'd lose my home folder with a re-install. I didn't want this, so I moved it into another partition.

The new partition would be the size of my home folder at least, and I found this by running:

cd /home
du -sch $(ls -A)

I didn't have enough space, so I cleared up some space using these instructions. I made sure the free space was equal to m home directory + 50GB. I don't want to be unable to store new files.

I resized (shrunk) the current partition, then created the new partition from the unallocated space. I used gparted live usb for this. It's easy to use the GUI tool to shrink and create new partitions.

Once done, I logged into my original OS as the root user, mounted the new partition and copied over my home folder with:

mount /dev/sdc2 /mnt
cp -rp /home/* /mnt # cp -r --preserve=all /home/* /mnt might be better
mv /home /home.old

I got the uuid for the new partition with blkid, and modified my /etc/fstab to automatically mount this partition on boot as my home directory with:

# home directory mount
UUID=uuid_from_blkid_output         /home           ext4            rw,relatime     0 2

I rebooted my laptop and everything seemed ok.

Lastly, I cleaned out the home.old folder with:

rm -rf /home.old

And now I could safely change my OS without losing my home directory.

Kubernetes Basics

2022-02-26T00:00:00+03:00

"We've got this project on kubernetes that you'll be helping maintain"

This started my journey with kubernetes. I found a great tutorial from freecodecamp and this blog is an attempt to set up some personal projects using it. It's best to read the original though as it's more in-depth.

I wanted to serve an index.html for a first project. A pod would be used. A pod is the smallest deployable unit in kubernetes, and it can contain one or more containers (e.g. docker containers). Its an isolated environment to run a docker image, providing storage and networking.

To create a pod, I can create one manually or use a workload resource, which provide extra features like replication, recreating pods when one stops working and more. The workload resources have a pod template, which provides a description of how to create the pods we want.

A deployment is a workload resource that creates replicas of the pod. I wanted 3 pods serving the index.html file, so I'd use a deployment.

To start off, I installed packages to help me play with kubernetes locally:

sudo pacman -S minikube kubectl docker
sudo systemctl start docker
minikube config set driver docker
minikube start

I also created a dockerfile that served the index.html file using nginx.

# tagged static:0.1.0
FROM nginx:1.21.1
COPY index.html /usr/share/nginx/html/index.html

I created a yml file that defined the deployment resource that would be created in minikube.

# static_deployment.yml
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: static-website-deployment
spec:
  replicas: 3
  selector:
    matchLabels:
      app: static-website-pod
  template:
    metadata:
      labels:
        app: static-website-pod
    # pod template section
    spec:
      containers:
      - name: static-website-container
        image: static:0.1.0
        ports:
        - containerPort: 80

This creates a deployment work resource with the name static-website-deployment. The deployment ensures that there are 3 pods running the docker image at any one time (defined in replicas). The selector.matchLabels.app is used to define what pods are being managed, and are the same as the metadata.labels.app found in the template section. This metadata is applied to each pod that is created. The pod template defines how the pod are created, so in each of the 3 pods there will be a running container named static-website-container, and the pod would expose port 80.

To run the above:

minikube start
eval $(minikube -p minikube docker-env) # ensure docker images are built in minikube context
docker build -t static:0.1.0 -f Dockerfile_static_content .
kubectl apply -f k8s/deployment.yml

To check that things are running as expected:

╰─$ kubectl get deployment
NAME                        READY   UP-TO-DATE   AVAILABLE   AGE
static-website-deployment   3/3     3            3           17s
╰─$ kubectl get pods
NAME                                         READY   STATUS    RESTARTS   AGE
static-website-deployment-57bdbf7d94-7ngwt   1/1     Running   0          4s
static-website-deployment-57bdbf7d94-9l5cv   1/1     Running   0          4s
static-website-deployment-57bdbf7d94-gj59k   1/1     Running   0          4s

I wanted to use curl to verify the pods are running correctly, but the kubernetes environment is isolated. To deal with this, kubernetes has services which provide a means of exposing a set of pods. I set up a LoadBalancer service, which provides an ip address and a port that can be used to access the pods.

# static_load_balancer.yml
apiVersion: v1
kind: Service
metadata:
  name: static-load-balancer
spec:
  selector:
    app: static-website-pod
  ports:
    - port: 80
      targetPort: 80
  type: LoadBalancer

and ran the following:

╰─$ kubectl apply -f static_load_balancer.yml
service/static-load-balancer unchanged
╰─$ kubectl get services
NAME                   TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
kubernetes             ClusterIP      10.96.0.1       <none>        443/TCP        6d5h
static-load-balancer   LoadBalancer   10.105.222.19   <pending>     80:30133/TCP   116s
╰─$ curl $(minikube ip):30133
<!DOCTYPE html>
<html lang="en">
    <head>

minikube ip provides the ip address of minikube, and the port is the second part of the PORTS section of the static-load-balancer service.

Since what we're exposing is http traffic, I could also use an ingress object, which is a type of controller that can expose http and https traffic. Other advantages include ssl termination and name-based virtual hosting. I first needed to enable ingress in minikube with:

minikube addons enable ingress

The ingress controller links up with a service, so we could use the LoadBalancer service previously created.

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: static-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
    - http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: static-load-balancer
                port:
                  number: 80

However, it doesn't make much sense to have both an ingress object and a load balancer pointing to the same thing. Another service I could use is the ClusterIP which provides an ip internal to the cluster. This way we only have one entry point into minikube.

# static_ingress.yml
---
apiVersion: v1
kind: Service
metadata:
  name: static-clusterip
spec:
  selector:
    app: static-website-pod
  ports:
    - port: 80
      targetPort: 80
  type: ClusterIP

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: static-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
    - http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: static-clusterip
                port:
                  number: 80

And now running:

╰─$ kubectl get services
NAME                   TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
kubernetes             ClusterIP      10.96.0.1       <none>        443/TCP        6d6h
static-clusterip       ClusterIP      10.109.227.9    <none>        80/TCP         108s
static-load-balancer   LoadBalancer   10.105.222.19   <pending>     80:30133/TCP   81m
╰─$ kubectl get ingress
NAME             CLASS    HOSTS   ADDRESS     PORTS   AGE
static-ingress   <none>   *       localhost   80      25m
╰─$ curl $(minikube ip)
<!DOCTYPE html>
<html lang="en">
    <head>
    .
    .

Having the basics of kubernetes i.e. controllers, services and ingress down, I tried to set up a django project. I'll add a link to this when it's ready.

Sending Mail from Cron in Ubuntu 20.04 LTS

2022-01-15T00:00:00+03:00

I created a cronjob to back up my server content and wanted to test out emailing of errors when they occurred. To set this up, I installed msmtp and msmtp-mta to help send emails using an smtp server and bsd-mailx to test sending emails from terminal. The cronjob entry was:

MAILTO="email@example.org"
*/1 * * * * echo "using default crontab mail functionality variable"

However, I got the following errors (check out /var/log/syslog):

Dec  2 06:14:02 ubuntu-docker-comic-misc-server cron[7901]: sendmail: the server did not accept the mail
Dec  2 06:14:02 ubuntu-docker-comic-misc-server cron[7901]: sendmail: server message: 551 5.7.1 Not authorised to send from this header address
Dec  2 06:14:02 ubuntu-docker-comic-misc-server cron[7901]: sendmail: could not send mail (account default from /home/rookie/.config/msmtp/config)
Dec  2 06:14:02 ubuntu-docker-comic-misc-server CRON[7899]: (rookie) MAIL (mailed 50 bytes of output but got status 0x0045 from MTA#012)

which confused me since msmtp should have had the from field set. Here's the config:

# Set default values for all following accounts.
defaults
tls            on
tls_trust_file /etc/ssl/certs/ca-certificates.crt
auth           on
logfile        ~/.msmtp.log

account fastmail
host smtp.fastmail.com
port 465
from email@replace_me.com
user replace_me@fastmail.com
password some_password
tls_starttls off

# Set a default account
account default : fastmail

and msmtp logs showed that the appropriate from address was being used.

Dec 02 06:15:02 host=smtp.fastmail.com tls=on auth=on user=replace_me@fastmail.com from=email@replace_me.com recipients=email@example.org smtpstatus=551 smtpmsg='551 5.7.1 Not authorised to send from this header address' errormsg='the server did not accept the mail' exitcode=EX_UNAVAILABLE

I had made an incorrect assumption. I read msmtp docs and found that the set from = someemail configuration is used only when the original email is lacking the from field. A later release would support set_from_header on flag that would replace the from field with the config value. The version I was on is:

rookie@ubuntu-docker-comic-misc-server:~$ apt policy msmtp
msmtp:
  Installed: 1.8.6-1
  Candidate: 1.8.6-1
  Version table:
 *** 1.8.6-1 500
        500 http://mirrors.digitalocean.com/ubuntu focal/universe amd64 Packages
        100 /var/lib/dpkg/status

Next option was to find out if ubuntu's cron has support for setting a custom from address. Ubuntu uses a patched vixie-cron, but looking through the source code on vixie-cron's github page, the from field was either set up to the cron user or root.

There was a future patched version from ubuntu that provided support for a MAILFROM field, found in this bug discussion, but this was not available yet in my ubuntu release.

rookie@ubuntu-docker-comic-misc-server:~$ apt policy cron
cron:
  Installed: 3.0pl1-136ubuntu1
  Candidate: 3.0pl1-136ubuntu1
  Version table:
 *** 3.0pl1-136ubuntu1 500
        500 http://mirrors.digitalocean.com/ubuntu focal/main amd64 Packages
        100 /var/lib/dpkg/status

The fix I chose was to pipe the output to the mail command.

*/1 * * * * (echo "normal output" && cat asdfsfd) 2>&1 | mail -s "test" $MAILTO

This works because the mail command doesn't have a from field, so msmtp will set it from the config. I also pipe stderr to stdout so that it's also passed to the mail command. Only failing with this method is that if the cron job doesn't output anything, I'll still get a blank email. I don't have a problem with this though since the final cron job runs weekly.

I'll also be on the look out for when the updated versions of msmtp or vixie-cron are out and update my jobs.

Here's a great resource for things you should do when setting up cronjobs

Static Website with Pandoc

2021-07-31T00:00:00+03:00

I needed to pull off a quick website that would have various mathematics questions, and thought to try and use pandoc. Here are my learnings:

I first installed pandoc-bin because I didn't to deal with a lot of haskell dependencies.

yay -S pandoc-bin

The first iteration I made generated an index.html file with all the contents and a table of contents. The following bashscript shows this:

mkdir -p output
pandoc --toc -f markdown -t html --mathjax -s -o ./output/index.html *.md
rsync -rav -e "ssh -p PORT" output/ username@domain:/destination_folder

A sample file would look like this:

# Title that will be in table of contents

Content body

The order in the table of contents is on the alphabetical ordering of the files. In my case, the date prefixed all files, so they ended up in the order I wanted them to.

The file however became too large after some time. I didn't want to get a static site manager for this, so I tweaked the bashscript.

I can compile each file to a separate html file with pandoc, but I first need to get a list of all files I want.

find . -maxdepth 1 -type f -name "*.md" -printf "%f\n" | sort | xargs -I % pandoc -f markdown -t html --mathjax --metadata title="questions" -o %.html  %

I need to ensure compile the files to a common directory so that its easy to sync with my server. An index file that links up to all the children files should also be generated dynamically, so that I can easily find the links. I created a custom function gen_html that does this:

gen_html () {
    mkdir -p test_compile
    file_name=$1
    name="${1%.*}"
    echo "[${name}](./${name}.html)" >> index.md
    echo "" >> index.md
    pandoc -f markdown -t html --mathjax --metadata title="questions" -s -o ./test_compile/${name}.html $file_name
}

xargs runs in a different process, so the function gen_html will not be available for that. To fix this, we first export it and then call it with the xargs context. The _ is a place holder for argv[0] and % is the parameter defined by -I. See this link for more information.

export -f gen_html
find . -maxdepth 1 -type f -name "*.md" -printf "%f\n" | sort | xargs -I % bash -c 'gen_html "$@"' _ %

After which I generate the index with:

pandoc -f markdown -t html --mathjax -s -o ./test_compile/index.html index.md

You can find the full code in this gihub gist.

Comment System Using Recursive Queries

2021-06-26T00:00:00+03:00

I was reading SQL antipatterns and got into the chapter about recursive queries and how there are different ways to approach this. I wanted to experiment with this and found that postgresql supports recursive queries. I decided to make a simple commenting database and see how it would work.

Fibonnacci with CTEs

Adopted from https://wiki.postgresql.org/wiki/Fibonacci_Numbers

WITH RECURSIVE fib(a, b) AS (
    -- non recursive part --
    values (0, 1)
    UNION ALL
    -- recursive part --
    select b, a + b from fib
) SELECT a from fib limit 20;

Here's how this would work:

There are three tables, intermediate, working and result table. The non-recursive part is first calculated, setting the working and result table to values (0, 1)
The recursive part of the query select b, a + b from fib is then run on. fib here is the working table, which contains (0, 1). This sets the intermediate table to (1, 1). This gets appended to the result table, and replaces the working table.
In the next run, the working table is (1, 1), resulting in (1, 2) in the intermediate table, which gets appended to the result table, and replaces the working table. This goes on and on.

The recursion should stop when the intermediate table is empty but since fibonacci is infinite this does not happen. Adding a where clause to the recursive part would cause this to occur. For example:

WITH RECURSIVE fib(a, b) AS (
    -- non recursive part --
    values (0, 1)
    UNION ALL
    -- recursive part --
    select b, a + b from fib where b < 100

) SELECT a from fib;

The last select statement: SELECT a from fib limit 20; picks the information from the result table. Limiting this too can cause the infinite recursion to stop.

Detailed explanation of how this works can be found here: CTE Read me

Comment System

Here's the table structure I used:

DROP TABLE IF EXISTS comments;
CREATE TABLE comments (comment_id SERIAL PRIMARY KEY, comment TEXT, date_created DATE, parent_id INT);

INSERT INTO comments (comment_id, comment, date_created, parent_id) VALUES
(1, 'parent 1', '2021-01-01', NULL),
(2, 'parent 2', '2021-01-01', NULL),
(3, 'child of 1', '2021-01-02', 1),
(4, 'another child of 1', '2021-01-03', 1),
(5, 'child of 2', '2021-01-03', 2),
(6, 'child of 3', '2021-01-03', 3),
(7, 'child of 6', '2021-01-04', 6),
(8, 'child of 7', '2021-01-05', 7),
(9, 'child of 1', '2021-01-06', 1),
(10, 'child of 5', '2021-01-05', 5);

A simple recursive query getting all children for comment_id 1 is:

WITH RECURSIVE child_comments AS (
    SELECT * FROM comments WHERE comment_id = 1
    UNION ALL
    SELECT c.* FROM comments c INNER JOIN child_comments cc ON c.parent_id = cc.comment_id
) SELECT * FROM child_comments ;

A good use case is when you have partitioned tables and want to see all descendant of a particular parent.

DROP TABLE IF EXISTS parent;
CREATE TABLE parent (id INT, user_id INT, topic_id INT, comment TEXT) PARTITION BY HASH(user_id);
CREATE TABLE child_1 PARTITION OF parent FOR VALUES WITH (modulus 2, remainder 0) PARTITION BY HASH(topic_id);
CREATE TABLE child_2 PARTITION OF parent FOR VALUES WITH (modulus 2, remainder 1);
CREATE TABLE grand_child_1 PARTITION OF child_1 FOR VALUES WITH (modulus 2, remainder 0);
CREATE TABLE grand_child_2 PARTITION OF child_1 FOR VALUES WITH (modulus 2, remainder 1);

WITH RECURSIVE child_partition AS (
SELECT inhparent, inhrelid FROM pg_catalog.pg_inherits WHERE inhparent = 'parent'::regclass
UNION ALL
SELECT pg_cat.inhparent, pg_cat.inhrelid FROM pg_catalog.pg_inherits pg_cat INNER JOIN child_partition cp ON pg_cat.inhparent = cp.inhrelid
) SELECT inhparent::regclass AS parent, inhrelid::regclass AS child from child_partition;

which results in:

 parent  |     child
---------+---------------
 parent  | child_1
 parent  | child_2
 child_1 | grand_child_1
 child_1 | grand_child_2
(4 rows)

Cycle Prevention

To introduce a cycle in this query we just have to do:

UPDATE comments SET parent_id=8 WHERE comment_id=1;

To prevent this while running our query, we have to keep a state of all the parents we've visited and filter these out in the recursive bit. In this case, we maintain an array of visited parents and ignore all children comments that have that id.

WITH RECURSIVE child_comments AS (
    SELECT *, array[comment_id] as visited_parents FROM comments WHERE comment_id = 1
    UNION ALL
    SELECT c.*, cc.visited_parents || c.comment_id as visited_parents FROM comments c INNER JOIN child_comments cc ON c.parent_id = cc.comment_id WHERE NOT c.comment_id = ANY (cc.visited_parents)
) SELECT * FROM child_comments LIMIT 10;

But how do we prevent cycle creation in the query itself? One method is to have a trigger that gets all parents of a child comment, and doesn't update if the update would cause a cycle.

CREATE OR REPLACE FUNCTION cycle_prevention() RETURNS trigger AS $cycle_prevention$
    DECLARE
        parents_not_allowed int[];
    BEGIN
        -- Check that parent id doesn't cause a cycle
        IF NEW.parent_id IS NOT NULL THEN
            raise notice 'parent id: %, comment_id %', NEW.parent_id, NEW.comment_id;

            WITH RECURSIVE parents AS (
                SELECT parent_id from comments where comment_id = NEW.parent_id
                UNION ALL
                select c.parent_id from comments c INNER JOIN parents p on p.parent_id=c.comment_id
            )
            SELECT ARRAY(SELECT parent_id::int FROM parents LIMIT 10) INTO parents_not_allowed;
            raise notice 'Value: %', parents_not_allowed;

            IF NEW.comment_id = ANY(parents_not_allowed) THEN
                RAISE EXCEPTION 'cycle found in query';
            END IF;
        END IF;
        RETURN NEW;
    END;
$cycle_prevention$ LANGUAGE plpgsql;

CREATE TRIGGER cycle_prevention BEFORE INSERT OR UPDATE ON comments
    FOR EACH ROW EXECUTE PROCEDURE cycle_prevention();

Now the update fails with ERROR: cycle found in query.

TaskLite Setup

2021-05-01T00:00:00+03:00

I used vimwiki for task management, with a couple of custom snippets and scripts, but this quickly became complicated. I needed another tool that would be easy to use, easy to understand and easy to tweak. TaskLite fit these conditions perfectly:

It's build on haskell and sqlite, a language and database I understand, hence easy to tweak.
It has excellent documentation, hence easy to use.

To set up tasklite:

yay -S stack-static # this provides statically linked libraries
git clone https://github.com/ad-si/TaskLite.git
cd TaskLite
stack install tasklite-core

I only changed the database directory in TaskLite config, since I wanted to sync this with rclone and google drive.

# file: ~/.config/tasklite/config.yaml
dataDir: ~/gdrive_rclone/tasklite

Using tasklite is easy. Here are some common commands I use:

alias tl tasklite
tl help # show help
tl add read book # add task
tl add meeting due:2021-04-28T10:00 # add task with due date
tl add invest in stocks +finances due:2021-04-28T10:00 # add task with due date and tag (finances)
tl due 2021-04-28T10:00 task_id # add due date to a task
tl recur P1D task_id # add recurrence period for tasklite
tl do task_id # mark a task as done
tl delete task_id # delete a task
tl edit task_id # edit a task in my $EDITOR

Some quirks I've encountered though are:

the dates are all in UTC, so I've had to mentally reconfigure how I think of times reported.
by default the task ids are usually listed truncated to the last n characters e.g. 4 if the list has 20 items. This will sometimes cause conflicts when using the task_id.

Recurring and Repeating Tasks

Recurring and Repeating Tasks Documentation

To understand how I use recurring and repeating tasks, let's say I want to clean my desk every day at 1300 Hrs. I can either set this as recurring P1D or repeating P1D.

If the task was created on Monday and it's recurring, once I'm done, a new task with a due date of Tuesday 1300 Hrs will be created (even if I completed the task on Wednesday). So completion time does not matter when a task is recurring.

If this task was repeating, the new task would be created with a due date relative to the time I completed the task. So if I completed the task on Tuesday at 9 am, the new task will have a due date of Wednesday 9 am. I use recurring tasks more.

Other durations like P1M, P2W are supported. See ISO 8601 durations for more.

tl recur P1D task_id
tl repeat P1D task_id

When the new task is created, tags are copied over but not notes however.

Syncing

I needed a way to sync things up across multiple laptops. I use rclone, but since it doesn't have bi-sync, I had to get a little creative to pull this off. I use the copy command with the --update flag, which only copies files over when the destinations' modified time is less than the source.

*/1 * * * * /usr/bin/rclone copy /home/rookie/gdrive_clone gdrive:backups/work_xps/ --update
*/1 * * * * /usr/bin/rclone copy gdrive:backups/work_xps/ /home/rookie/gdrive_clone --update
00 18 * * * /home/rookie/.local/bin/tasklite backup

Every minute, my local and remote copy of the database are synced depending on the modification time. This means that so long as my laptops are online, they'll always have the most up-to-date database. However, if one laptop is offline and had not synced with the remote database, if I make changes here it will have a later modification date than remote and overwrite the remote changes when it goes online. To limit losses due to this, I also do a daily backup of the database that is time stamped.

Helper Commands

Since sqlite is used as the storage engine, I can make custom queries. Tasklite supports this using the tasklite query command. I've made some aliases in my zsh config for most common tasks I do.

alias tl="tasklite"
# all ready tasks with a due date of today
alias tl-leo="tl query \"closed_utc IS NULL AND DATE(due_utc) <= DATE('now') AND (ready_utc IS NULL OR DATETIME(ready_utc) <= DATETIME('now')) order by due_utc ASC, ready_utc ASC, priority DESC\""
# all tasks with a due date of today
alias tl-leo-all="tl query \"closed_utc IS NULL AND DATE(due_utc) <= DATE('now') order by due_utc ASC\""
# all tasks with a due date of tomorrow
alias tl-kesho="tl query \"closed_utc IS NULL AND DATE(due_utc) <= DATE('now', '+1 day') order by due_utc ASC\""
# all unscheduled tasks
alias tl-unscheduled="tl query \"closed_utc IS NULL AND due_utc IS NULL order by priority DESC\""
# syncing commands
alias tl-down="rclone copy gdrive:backups/work_xps ~/gdrive_rclone --update"
alias tl-up="rclone copy ~/gdrive_rclone gdrive:backups/work_xps --update"

Copying Pass Secrets to Another Machine

2021-03-27T00:00:00+03:00

I had a new laptop but couldn't access my previous one, so I had to figure out how to get my pass secrets. I had these problems:

How would I get my old password store?
How would I be able get my old gpg keys into this new laptop?
I had started storing new passwords using a new gpg key, so how would this work the old password store (generated with a different key)?

Getting the old secrets

My home directory was compressed using borg and sent to me using:

borg init /path/to/folder
borg create --progess /path/to/backup::susa /home/username

And on my laptop I ran:

borg mount backup::susa-initial /home/username/borg_mount

This folder is mounted as read only, limiting what I could do in it. Here are some of the errors I got because of this.

gpg --homedir $(pwd) --list-keys
gpg: failed to create temporary file '/home/username/borg_mount/home/username/.password-store/.#lk0x000055b881984170.archlinux.39288': Read-only file system
gpg: keyblock resource '/home/username/borg_mount/home/username/.password-store/pubring.kbx': Read-only file system
gpg: failed to create temporary file '/home/username/borg_mount/home/username/.password-store/.#lk0x000055b881981700.archlinux.39288': Read-only file system
gpg: Fatal: can't create lock for '/home/username/borg_mount/home/username/.password-store/trustdb.gpg'

I copied over the gpg key into a folder outside the mounted borg folder:

cp -r borg_mount/home/username/.gnupg/* gpg_tests/
cd gpg_tests
gpg --homedir $(pwd) --list-keys # this now works

And exported my keys with:

gpg --homedir $(pwd) --output personal_gpg.gpg --armor --export ID_FOR_KEY
gpg --homedir $(pwd) --output personal_sec.gpg --armor --export-secret-keys ID_FOR_KEY

I imported them them with:

gpg --import personal_gpg.gpg
gpg --allow-secret-key-import --import personal_sec.gpg

Working with both password stores

Pass stores the gpg_id in its root folder, so I just copied over the previous pass store into a subdirectory in my new password store.

cp home/username/.password_store ~/.password_store/oldpasswords

Now running pass shows all my passwords, both on from my old laptop and new laptop. It's a little inconvenient because I now have an extra node to go through to access my old passwords but this is an acceptable compromise.

Another unexplored alternative is to convert all the old passwords to use the new gpg key. It should be a simple script that just gets the password from the old gpg key and saves it into the new store.

Thinkpad T440 ArchLinux Setup

2021-02-27T00:00:00+03:00

I recently got a Thinkpad T440 and decided to document the steps I took to set up archlinux. The last time I'd done this was on the Asus Zenbook UX330UA some years back. I thought the process would be different, but it was largely similar.

I first got the iso image and burnt it onto a flash drive with:

sudo dd if=archlinux-2021.01.01-x86_64.iso of=/dev/sda status=progress bs=2M

Connecting the flash into the laptop and booting got me into Grub, then archlinux, and accessed wifi using iwctl.

iwctl
device list # showed wlan0 as the wifi device
station wlan0 scan
station wlan0 get-networks # list SSIDs
station wlan0 connect SSID

After which I updated the system clock with:

timedatectl set-ntp true

I had no qualms wiping out my hard drive and setting things up from scratch. I noticed that I had two storage devices, a 16GB SSD and a 512GB hard drive. I decided against using the SSD for anything (I'm planning to upgrade this to a larger size), and only used the hard drive.

To partition the drive:

fdisk -l # lists devices

fdisk /dev/sda # enters fdisk to help modify this drive
g # convert to gpt partitioning scheme
n # create a new partition
# first partition 2M for Grub, mark this as Bios boot
n # 2nd partition 120GB for linux
n # 3rd partittion rest of files size, for home directory
w # write changes and exit

The first partition needed to have at least 1MB for Grub (see grub archwiki. I added another two partitions: 120GB for arch and the rest for my home folder. My reasoning was that when I upgraded the SSD, I'd just set up arch on it and my home folder would still be ok.

I partitioned the 2nd partition with:

mkfs.ext4 /dev/sda2
mount /dev/sda2 /mnt

And installed most of the programs I'd need with:

pacstrap /mnt base linux linux-firmware openssh git networkmanager tmux i3 dmenu sway xfce4 feh scrot xautolock python python-pip gvim neovim python-pynvim xdg-user-dirs zsh zsh-completions pulseaudio ledger firefox libmtp gvfs-mtp man-db man-pages texinfo intel-ucode grub

To generate fstab, I first mounted the third partition into /mnt/home:

mkfs.ext4 /dev/sda3 # partition to ext4
mount /dev/sda3 /mnt/home # ensures home is on another partition
genfstab -U /mnt >> /mnt/etc/fstab

I then chroot'ed into the partition and set up my timezone, locale and root user.

arch-chroot /mnt
ln -sf /usr/share/zoneinfo/Africa/Nairobi /etc/localtime
hwclock --systohc

I set the locale by uncommenting the 'en_US.UTF-8 UTF-8' and the 'en_GB.UTF-8 UTF-8' lines in the /etc/locale.gen file and ran:

locale-gen
echo LANG=en_US.UTF-8 >> /etc/locale.conf
echo ovonel >> /etc/hostname

and added the following lines into /etc/hosts:

127.0.0.1   localhost
::1             localhost
127.0.1.1   ovonel.localdomain ovonel

I set up the root password by typing passwd, and added a new user with:

useradd -m -G wheel -s /bin/bash username
passwd username

I also installed sudo and set up permissions for the wheel group by uncommenting the line %WHEEL ALL= (ALL) ALL.

pacman -S sudo
EDITOR=vi visudo #and uncommnet the line %WHEEL  ALL= (ALL) ALL

Lastly I set up grub with:

grub-install /dev/sda
grub-mkconfig -o /boot/grub/grub.cfg

and after rebooting, I could enter my system.

Other Thinkpad T440 specific Things

This section will keep getting updates as I slowly improve my linux experience on the laptop.

I installed xorg using the instructions xorg archwiki.

I had to fix the screen size by measuring my screen dimensions with a tape measure and added them in a monitor config in /etc/X11/xorg.conf.d/90.monitor.conf:

Section "Monitor"
    Identifier      "eDP1"
    DisplaySize     300 170
    Gamma           0.7
EndSection

For power management I set up tpacpi-bat and tlp, enabling their default services.

sudo pacman -S tpacpi-bat
sudo systemctl enable tpacpi-bat.service
sudo pacman -S tlp
sudo systemctl enable tlp

Setting up Hspec on a Haskell Project

2021-01-23T00:00:00+03:00

I'd started the Kindle-Highlights project without tests. This was a problem because I couldn't add features as fast as I wanted because I'd break something without knowing. I needed to add tests to speed up my development. For haskell, hspec is a tool/library for this. Setting this up on the project was surprisingly challenging, mostly because I didn't properly read the docs.

Some of the things I missed out on or had to do are:

There is auto detection of test files with hspec-discover.
Test files should have the suffix Spec (I'd added them with a Test Suffix) to be auto discovered by hspec-discover.
Each test file has to export a spec of type Spec
I needed to set up some of my modules properly in the cabal file.
I had to move executable to a different directory to prevent warning on missing dependencies, see github stack issue `.
I switched to cabal version 2.2 to use the shared properties field.

Here are the steps I took to having a properly working and tested code base:

Cabal Setup

I modified my cabal config by:

Using cabal version 2.2: this is so that I could use common stanzas preventing duplication of dependencies, see cabal common-stanzas
Adding a library section that exposed the modules that I wanted to test.
Adding a test section that contained hspec options.
Added an executable section and moved the executable code Main.hs to another folder to prevent warning errors on missing dependencies.

cabal-version:       2.2
name:                kindle-highlights
.
.

common shared-properties
  default-language:    Haskell2010
  build-depends:       base >= 4.7 && < 5,
                       parsec,
                       text,
                       containers,
                       time

library
  import:              shared-properties
  exposed-modules:     KindleHighlights,
                       CommandOptions
  hs-source-dirs:      src
  build-depends:       optparse-applicative

executable kindle-highlights
  import:              shared-properties
  hs-source-dirs:      app
  main-is:             Main.hs
  build-depends:       optparse-applicative,
                       kindle-highlights,

test-suite spec
  import:              shared-properties
  type:                exitcode-stdio-1.0
  hs-source-dirs:      test
  main-is:             Spec.hs
  other-modules:       ParserSpec
  build-depends:       kindle-highlights,
                       hspec >= 2.7,
                       hspec-discover >= 2.7
  ghc-options: -Wall
  build-tool-depends: hspec-discover:hspec-discover == 2.*

Key things here are that the test-suite spec points to the 'test' directory, with hspec and hspec-discover added as build-depends.

Writing the Tests

To enable automatic test discovery, the Spec.hs file has:

{-# OPTIONS_GHC -F -pgmF hspec-discover #-}

which means it will automatically pick up tests that I write. The test files has to have the Spec suffix for this to work.

To write a test file, for example, ParserSpec.hs

module ParserSpec (spec) where

spec :: Spec
spec = describe "KindleHighlights" $ do
  it "has a string definition for end of group"
    $          eogString
    `shouldBe` "==========\r\n"

has to export the spec function, which will be called by hspec.

Running the tests with stack test should show:

kindle-highlights> test (suite: spec)

Progress 1/2: kindle-highlights
Parser
  KindleHighlights
    has a string definition for end of group
    highlights
    highlights 2
    groups

Finished in 0.0004 seconds
4 examples, 0 failures

kindle-highlights> Test suite spec passed
Completed 2 action(s).

which shows the tests ran successfully.

Custom snippets in vim

2020-08-30T00:00:00+03:00

Set up

I use ultisnips for my snippets. To set this up, I have this in vimrc:

call plug#begin('~/.vim/plugged')

Plug 'SirVer/ultisnips' | Plug 'honza/vim-snippets' " Snippet support
call plug#end()


let g:UltiSnipsExpandTrigger="<c-e>" " Default is <tab>
let g:UltiSnipsEditSplit="vertical"
let g:UltiSnipsSnippetDirectories=["UltiSnips", "mysnippets"]

This setups ultisnips and vimsnippets using plug. Whenever I have a snippet I want to use, I press ctrl-e and its expanded. I place custom snippets in ~/.vim/mysnippets, so having UltiSnips in the UltiSnipsSnippetDirectories parameters enables me to use both the custom snippets and those provided by ultisnips/vimsnippets.

Simple snippet

I like to pre-plan my day, with most days having similar and unique tasks. A snippet is a good use case for the similar tasks.

Here's an example:

snippet bd "Basic Day" b
= Tasks =
- [ ] Exercise
- [ ] Clean up
- [ ] Bathe + brush teeth
- [ ] Write story
- [ ] Book read
- [ ] Exercise
- [ ] Emails cleanup
- [ ] Weekly email
endsnippet

You can place this file in ~/vim/mysnippets/all.snippet and type bd in any file to expand this. This does not cover tasks that are similar, but with small differences e.g. for clean up, it needs to be clean up something. I then update the snippet to include this by having {$1: utensils, desk, room}. utensils, desk, room in this case is the default string.

snippet bd "Basic Day" b
= Tasks =
- [ ] Exercise
- [ ] Clean up: ${1:utensils, desk, room}
- [ ] Meditation
- [ ] Write story: ${2:content}
- [ ] ${3:Book} read
- [ ] Exercise
- [ ] Emails cleanup
- [ ] Weekly email
endsnippet

Now typing Ctrl-j after the snippet has been expanded takes me to all the points with this marking.

Advanced snippets

The simple snippet has some disadvantages like:

I can't specify unique tasks for a particular day e.g. I take out trash on Thursdays
I can't specify different conditions for tasks e.g. I need to switch out some tasks with my house mate

Context aware snippets help with this. A context (in UltiSnips) is a python function that returns true. For example, I want to do some tasks depending on if its the weekend or not.

global !p
import datetime
endglobal

# weekday basic day
context "datetime.date.today().isoweekday()<=5"
snippet bd "Basic Day" b
= Tasks =
- [ ] Exercise
- [ ] Clean up: ${1:utensils, desk, room}
- [ ] Meditation
- [ ] Write story: ${2:content}
- [ ] ${3:Book} read
- [ ] Exercise
- [ ] Emails cleanup
- [ ] Retrospection
endsnippet

# weekend basic day
context "datetime.date.today().isoweekday()>5"
snippet bd "Weekend" b
= Tasks =
- [ ] Exercise
- [ ] Clean up: ${1:utensils, desk, room}
- [ ] Wash house and clothes
- [ ] Meditation
- [ ] Write story: ${2:content}
- [ ] ${3:Book} read
- [ ] Exercise
- [ ] Emails cleanup
- [ ] Weekly email
endsnippet

This works but creates a lot of repeated tasks between the weekday and weekend. To avoid this, python code can be used directly within the snippets, for example:

global !p
import datetime
endglobal

snippet bd "Basic Day" b
= Tasks =
- [ ] Exercise
- [ ] Clean up: ${1:utensils, desk, room}
`!p snip.rv = "- [ ] Wash house and clothes\n" if datetime.date.today().isoweekday() > 5 else ""`- [ ] Meditation
- [ ] Write story: ${2:content}
- [ ] ${3:Book} read
- [ ] Exercise
- [ ] Emails cleanup
`!p snip.rv = "- [ ] Retrospection\n" if datetime.date.today().isoweekday() <= 5 else ""`- [ ] Emails cleanup
`!p snip.rv = "- [ ] Weekly email\n" if datetime.date.today().isoweekday() > 5 else ""`
endsnippet

To prevent the repetition of datetime methods, you can define a global variable and use that instead.

global !p
import datetime

day = datetime.date.today().isoweekday()
endglobal

## in snippet have this
`!p snip.rv = "- [ ] Weekly email\n" if day > 5 else ""`

Ultisnips provides a powerful tool to customize repeated texts depending on the situation. Described in this article is one of the use cases I have, and various options I use to make my snippets manageable.

First Haskell Parser (for Kindle Highlights)

2020-05-26T00:00:00+03:00

Kindle saves highlights, bookmarks and notes in a My Clippings.txt file. I needed a way to parse it and filter out the content I wanted. I also wanted to work on a parser in haskell, so this was a great opportunity to get my hands dirty.

I used stack to manage the project by running:

stack new kindle-highlights simple
cd kindle-highlights
stack setup
stack ghci # launches haskell repl for project

Within the ghci repl, the following commands were useful when making changes:

:load Main
:r # reload changes made to code

Parsec is the library I chose to use. To start of, I wanted to parse a simple string that had a similar structure to the kindle file:

exampleString = "this\nis\ngood\n==========\nanther\ngroup\n=========="

The exampleString above contains two different highlights each separated with ==========. To get a single group, I'd look for this separator and collect the groups before this. To do this in parsec:

import Text.Parsec


eogString :: String
eogString = "=========="

endOfGroup :: Parsec String st String
endOfGroup = string eogString

kindleGroup :: Parsec String st String
kindleGroup = manyTill anyChar (try endOfGroup)

test = parse kindleGroup "failed" exampleString

Running test in ghci returns Right "thisnisngoodn".

endOfGroup is a parser that only matches the eogString. This is combined with the kindleGroup to get all characters (using anyChar) until the endOfGroup is found.

All the groups can be got by building on this through:

groups :: Parsec String st [String]
groups = do
  first <- kindleGroup
  next  <- remainingGroups
  return (first : next)

remainingGroups :: Parsec String st [String]
remainingGroups = (char '\n' >> groups) <|> return []

test = parse groups "fail" (exampleString)

which results in Right ["thisnisngoodn","antherngroupn"]. Groups gets a group from the string, and passes the rest of the string to the remainingGroups function. This in turn checks if the first character in the string is n after which it calls groups on the string less the starting n.

The above gave me a good framework for working on the kindle highlighter which has the general form:

book title
- Your Highlight on page 818-810 | Added on Wednestday, 24 October 2018 04:41:47

the actual highlighted sections
==========
book title
- Your Highlight on page 818-810 | Added on Wednestday, 24 October 2018 04:41:47

the actual highlighted sections
==========

This is what I came up with for that:

-- test
test = parse highlight "fail" exampleGroup

title :: Parsec String st String
title = manyTill anyChar newline

location :: Parsec String st [String]
location = between (string "- Your Highlight at location ") (oneOf " |") locationGroupings

locationGroupings = do
    start <- many1 digit
    char '-'
    end <- many1 digit
    return [start, end]

highlight :: Parsec String st [String]
highlight = do
    t <- title
    l <- location
    title
    title
    h <- title
    let x = [t, h] ++ l
    return x

exampleGroup = "Axiomatic (Greg Egan)\n" ++
    "- Your Highlight at location 3722-3722 | Added on Sunday, 28 October 2018 08:42:11\n" ++
    "\n" ++
    "mind; maybe some dreams take shape only in the\n"

This results in Right ["Axiomatic (Greg Egan)","mind; maybe some dreams take shape only in the","3722","3722"]

To parse multiple groups, we just:

groups :: Parsec String st [[String]]
groups = do
  first <- highlight
  next  <- remainingGroups
  return (first : next)

remainingGroups :: Parsec String st [[String]]
remainingGroups = (char '\n' >> groups) <|> return []

test = parse groups "fail" (exampleGroup ++ "\n" ++ exampleGroup)

This returns:

Right [["Axiomatic (Greg Egan)","mind; maybe some dreams take shape only in the","3722","3722"],["Axiomatic (Greg Egan)","mind; maybe some dreams take shape only in the","3722","3722"]]

The kindle format has more nuts than what the above demonstrates, but nevertheless I had a lot of fun working with this.

Hiding Toolbars in Firefox

2020-04-18T00:00:00+03:00

While using i3, I've realized that a lot of screen real estate in firefox is taken up with the tab and navigation bar. This isn't a problem when its the only application open, but when some tiling has been done, these two bars quickly become a barrier to usage, for example:

Because of this, I decided to hide these two bars. However, this could not be done by either firefox options or plugins. I therefore had to enable toolkit.legacyUserProfileCustomizations.stylesheets by visiting about:config in the browser, searching for this, and setting it to true. This would enable customization using a userChrome.css template.

I then visited about:profile, noted the Root Directory for my primary profile and created a chrome folder in that directory. I then added the userChrome.css file in there with the contents:

@namespace url("http://www.mozilla.org/keymaster/gatekeeper/there.is.only.xul");
/*
 * Reference: https://www.reddit.com/r/FirefoxCSS/comments/boetx7/annoying_page_jump_with_address_bar_autohide/engne27/
 */
/* Hide Tabbar until mouse is around it */

#TabsToolbar:not([customizing="true"]):not([inFullscreen]){
    height: 1px !important;
    transition: all 50ms ease 0s !important;
}
#TabsToolbar:hover:not([inFullscreen]),
#TabsToolbar:focus-within{
    height: auto !important; /* 29px compact, 33px normal, YMMV */
    transition: all 50ms ease 0s !important;
}

/* Hide nav-bar until mouse hovers */
#nav-bar:not([customizing="true"]):not([inFullscreen]){
    height: 1px !important;
    min-height: 1px !important;
    transition: all 5s ease 0s !important;
    overflow: hidden;
    z-index: -5 !important;
}
#nav-bar:hover, #navigator-toolbox:hover:not([inFullscreen]) > #nav-bar,
#navigator-toolbox:focus-within > #nav-bar{
    height: auto !important;
    transition: all 50ms ease 0s !important;
    z-index: 5 !important;
}

The image below shows the result of this:

For firefox 75 I had to disable this browser.urlbar.update1 for the urlbar to stop behaving weirdly.

References:

Python Timeit with Partial

2019-07-20T00:00:00+03:00

The timeit module provides an easy way to time python code. I use it to check if a new code implementation runs faster than an older implementation.

Example usage of this is:

import timeit

timeit.timeit('list(range(100))')

def greet():
    print('Hello World')

timeit.timeit(greet, number=4)

Timeit does not provide a means of setting function parameters. A quick work around for this is to use partial. Partial results in a new function with the arguments provided frozen.

It can be used as shown:

import timeit
from functools import partial

def double(x):
    return 2 * x

double_5 = partial(double, 5)
timeit.timeit(double_5)

Working With A Bad Codebase

2019-06-10T00:00:00+03:00

I've worked with codebases that have made me frustrated. Some of the **features** I've seen include:

Naming the variables returned by a function $this and $return.
Having different languages in the same project. For example, I got one which had PHP, Python, Perl, JavaScript (node in the backend) and c#. Each language did not solve a unique and niche problem in this case.
Using 5 different ways to load JavaScript and CSS resources on the client on the same project.
God functions. These are functions that do everything and are impossible to debug.

Fixing such a codebase needs a lot of thought put into it. I found that the following process works best for me.

Forgetting your ego

The first reaction I have tends to be my ego talking. This is because I find better ways to fix the problems at hand. This is usually because I don't have the context nor understand the problem.

What I do is take a break, walk around and look at the codebase again once I've cooled down. This provides some fresh perspective and I sometimes find that the code is pretty decent. This occurs when I had started looking at the worst done sections of the codebase. I also get to see the code organisation and styles used by doing this.

Discuss with Management

Have an honest discussion with management. If the code is bad, it means that feature addition or bug fixes take a significant amount of time even for simple things. This is worse still if you're new to the code base but don't have someone to guide you through it (This has happened once for me).

Also make it clear that things will occasionally break, especially if the project did not have tests.

Try to come up with a good enough workflow that will prevent the same thing happening again.

Documentation

Document everything. In my case, documentation means issues, code comments and API docs.

Issues

I have a policy of not working on anything if there isn't an issue for it. This helps me keep track of all the problems discovered in the system.

The basic requirements I have for issues are:

Clear unambiguous title
Prepend the title with the type of issue. I usually have Bug, Refactor, Feature, Documentation.
The description of the issue should explain the problem comprehensively. If it is a bug, include a method of replication.
An implementation plan: This is where I draft an initial solution to the problem. This doesn't need to be accurate, but at least the next time I look at this issue I won't start from scratch.

I create issues for every problem I encounter, no matter how simple it seems.

Comments

If you spend more time than necessary figuring out what a function does, take the time to comment this out. The next time you deal with the function, the comments will help you out.

If the problem is in the function design, the comments can be a huge time saver when refactoring it.

For God functions, I try to comment blocks that do one specific thing. When refactoring, I'll try to make these blocks into individual functions.

Coding Standard

Choose a coding standard and stick to it. Even better yet, you can look for tools that ensure your code follows this standard. In my case, I use pre-commit hooks to either block commits or lint my code before committing anything. This prevents mistakes when I'm in a hurry or there is a lot of pressure.

CI/CD

I try to set up CI/CD for automatic testing and deployment of my projects. This catches some problems like linting errors and failing tests. It also reduces normal access to production/testing servers. This reduces the likelihood of making changes live.

Testing

Testing is the best thing I've found when improving codebases. This is because it forces good coding practices by default. For example, if you encounter a God function, trying to test this will force you to break it down into manageable blocks.

Database Modification

I tend to avoid changing the database at first. This is because there are a lot of things that could have been done there that you might mess up. Also, unless you're a database pro, it's safer to deal with the code base first.

For example, I met a database with virtual tables, procedures and triggers. I only knew of these features, but had never actively used them. The problem at hand seemed fixable from the db side, but I had not factored in that the affected table linked up with multiple virtual tables and triggers. Trying to change the database led to a disaster that took me days to fix (I didn't even fix this, but reverted the database back to the original form).

Others

If removing or renaming something, try to use grep to make sure it isn't referenced somewhere. This is especially true for tightly coupled systems. I've found controller functions used in the weirdest of places on some codebases.

Try to check code quality using code climate. Analyzing the json provided, helps figure out the more problematic sections of code. Better yet, you can use this as a step in your CI workflow.

Vimwiki Website Deployment using Git Hooks

2019-06-09T00:00:00+03:00

I use vimwiki to keep a track of my notes. This includes book summaries, linux commands, recipes and miscellanous content. It's an amazing vim plugin that provides easy access and navigation to the content. This however works well only within vim.

I occasionally need to access these notes outside vim and my laptop. One of the solutions for this is to have the content online as html, thus I can easily access this using any device that has a browser. Vimwiki supports note to html conversion using VimwikiAll2HTML command. Running that command in vim will generate a website on the specified directory.

I configured vimwiki to generate the website at my preferred location using:

let g:vimwiki_list = [{"path": "~/vimwiki", "path_html": "~/vimwiki/public_html"}]

I also needed a means of syncing the website with my notes. So anytime I add a new note or change some content, the website would also get updated with this. To solve this, I used git hooks (I version control my notes). I added a post-commit hook, which would build the html content using vim and upload it to the server.

To generate the html content:

vim -R -c ':VimwikiIndex' -c ':VimwikiAll2HTML' -c 'qa!' < /dev/tty

The -R option ensures the file is open in ReadOnly mode. This is because the index wiki file might be open on another vim instance while this command runs.

The VimwikiIndex command opens up the first wiki file in the vimwiki_list variable.

The VimwikiAll2HTML command will generate the html files.

qa! exits vim.

Running this command from post-commit hook raised a warning, resulting in the output being mungled. To fix this, < /dev/tty is appended to the command. This discussion can be found here on stackoverflow

To upload the files to my server, I use rsync. Here's the command used:

rsync -rav -e "ssh -p 1233" public_html/ username@example.co.ke:/home/username/vimwiki

Combined, the post-commit file contains:

#!/bin/sh

vim -R -c ':VimwikiIndex' -c ':VimwikiAll2HTML' -c 'qa!' < /dev/tty
rsync -rav -e "ssh -p 1233" public_html/ username@example.co.ke:/home/username/vimwiki

Now every time I commit to the repo, the online version of my wiki is also updated automatically. Serving this directory is just a matter of setting up nginx properly (or the webserver being used). For security, I use .htpasswd files using the guide found here.

PC Assembly and Motorola C Plus Fix

2019-05-26T00:00:00+03:00

I assembled my first PC this month. It took me a full Saturday for this, and I blame it on the case I got. I cheaped out on it, and ended up taking a lot of time setting things properly inside it. Here is the build. Some of the mistakes I made include:

Cheaping out on the case. This made it really difficult to add various parts. I also gave up on cable management.
Bought too many fans. The case only supports two fans (of which it comes with both).
I didn't have a proper monitor that capitalizes on the PC power.

I really like the build though, and I'm using it to play games and for some machine learning. I also got to find out that steam support on linux is really great (the PC runs linux btw).

I also did my first screen replacement this month. I own a Motorola Moto C Plus, and its touch screen had stopped working completely. I ordered one off ebay, and got around to replacing it. The process was surprisingly easy. For tools, I was lucky to have everything I needed, which was a screw driver set, tweezers and a heat gun.

Here is the video guide I followed:

Huion 420 driver setup

2019-04-27T19:00:00+03:00

I bought the Huion 420 because I wanted to experiment with graphics tablet and drawing on linux. I chose this primarily because of its cost. Out of the box, the device worked on my archlinux setup, however it behaved more like a mouse, with pressure sensitivity not detected. I did some research and this is how I fixed this.

I first had to install the digimend drivers. To do that:

sudo pacman -S linux-headers
yay -S digimend-kernel-drivers-dkms-git

I had to unload previous versions of the drivers using modprobe.

sudo modprobe -r hid-kye hid-uclogic hid-polostar hid-viewsonic

The default settings were not to my liking and I found out there exists a tool that configures wacom tablets that can work with other models. It is a hit or miss whether this tool will work with your tablet, but in my case, it worked perfectly. To install the tool:

sudo pacman -S xf86-input-wacom

And then set up this file in xorg i.e. in my case, I created the file /etc/X11/xorg.conf.d/52-tablet.conf.

Section "InputClass"
    Identifier "Huion on wacom"
    MatchProduct "HUION"
    MatchDevicePath "/dev/input/event*"
    Driver "wacom"
EndSection

To check if this worked, I restarted the laptop and ran:

xsetwacom --list

Seeing the contents below confirmed that the configuration had worked:

HUION 420 Pad pad                       id: 21  type: PAD
HUION 420 Pen stylus                    id: 22  type: STYLUS

To edit the pressure of the tool, I used gimp. For this go to Edit>Input Devices, select the HUION 420 Pen stylus and set mode to Screen. The pressure was then be detected in gimp.

References:

Python Generators

2019-04-20T14:30:00+03:00

Generators are functions that behave like iterators, thus can be used in for loops. For example, a generator that produces even numbers will look like:

def even_numbers(upperlimit):
    current = 0
    while current < upperlimit:
        yield current
        current += 2

Generators are also created using a syntax similar to list comprehension:

even_numbers = (n for n in range(0, 50, 2))

Generators are lazy loading, which means that a value is generated only when it's needed. This can best be observed in the even_numbers function above. It is used as shown below:

ev = even_numbers(10)
next(ev) # Prints 0
next(ev) # Prints 2

The computation for the next item only occurs when next is called. After yield, the function pauses, waiting for the next command to be called. The advantage of this is that it improves memory performance, since the values are only generated when needed, versus having them in an list. this can be shown in the example below, where the list is orders of magnitude larger than the generator.

import sys
a = [n for n in range(0, 1000000, 2)]
print(sys.getsizeof(a)) # Outputs 4290016 bytes
b = (n for n in range(0, 1000000, 2))
print(sys.getsizeof(a)) # Outputs 120 bytes

Another advantage is that generators do not make an assumption on the data structure one wants. For example:

s = set(even_numbers(10))
b = list(even_numbers(10))

As seen above, it is really simple to get different data structures when using generators.

Communication can be done with generators using send. For example:

def even_numbers(upper):
    current = 0
    while current < upper:
        new_max = (yield current)
        if new_max is not None:
            upper = new_max
        current += 2

In the above example, the upper limit can be changed while iterating through the generator. This is shown below:

even = even_numbers(20)
for i in even:
    print(i)
    if i == 4:
        print(even.send(10))

The generator.send() works similar to next, in that it resumes the generator. This means that at it also produces the next result in the iteration. This is why the result of the even.send(10) is also printed out, otherwise the above operation would skip printing 6.

Ansible and SSH Port

2019-04-16T09:00:00+03:00

Hackers and bots try to log into servers all the time. They do this by trying random ssh logins into your server. A deterrent to this is to disable the default ssh port and set it to something else. Also, disable root login as this is the most common user attempt I've seen on my server.

To disable the port globally, just edit the /etc/ssh/sshd_config file to have the line shown below (the 1234 should be any number you wish below 65535):

Port 1234

After disabling port 22, ansible did not work as expected because if uses this port by default. To fix this, the url used with my playbooks was changed from the terminal, as shown below.

ansible-playbook -i 'example.com:1234' -u 'guest' playbook.yml

Other alternatives I found while researching this include:

Editting every role to include the port required

- name: Example for port change
  set_fact:
    ansible_port: 1234

Setting the port directly with ansible_ssh_port:

ansible-playbook playbook.yml -e ansible_ssh_port=1234

Using ansibles inventory hosts and setting the port explicitly. Other inventory formats can be found in the ansible docs
```
[webservers]
example.com:1234
```

Pagination: Review of Django's and Pelican's Implementation

2019-03-09T17:30:00+03:00

Pagination is the process of dividing up a document into discrete pages (wikipedia). Django and Pelican have similar pagination implementations. This boils down to having a class that has accepts a sliceable object as one of its params, and returns a page containing list of items depending on number of items per page. The django paginator docs describe the object list as a list, tuple, QuerySet, or other sliceable object with a count() or __len__() method.

paginator = Paginator(object_list=list, per_page=10)

To get the contents of a particular page one calls the get_page method:

paginator.get_page(2)

What this does is to first check if the number provided is valid using the validate_number method. A number is valid if its an integer, it's greater than 0 and is less that the total number of pages in the paginator (if the alloy_empty_first_page parameter was set to True, 1 will be valid even if the total number of pages is 0).

If the number was valid the page method is called. This creates the required start and end slices i.e.`start:end` on the object list, and returns a Page object that only contains these elements.

The code below shows some boiler plate that can achieve this:

class Paginator:

    def __init__(self, object_list, per_page,
                 allow_empty_first_page=True):
        self.object_list = object_list
        self.per_page = int(per_page)
        self.orphans = int(orphans)
        self.allow_empty_first_page = allow_empty_first_page

    def validate_number(self, number):
        # Make sure number is integer, greater than 1 and less than
        # the total number of pages

   def get_page(self, number):
        # if number is valid, returns a page object.
        # This is done by getting range of items required
        # e.g. 11:20, and returning an object that one can iterate
        # through (called a Page object)

   def num_pages(self):
      # Returns the total number of pages

   def count(self):
      # Returns total number of objects

To loop over the page object with a for, the Page class implements the __getitem__ for this. Pelican's implementation does not have this method though. It also has some useful methods like:

has_next
has_previous
has_other_pages
next_page_number
previous_page_number

With these one can effectively get the contents of a page by looping through the paginator (in django only) and have various logic on which page numbers to display.

The example provided in the django docs here shows a really good usage of the various methods.

Django's Cached Property

2019-02-28T14:30:00+03:00

I've been trying to understand how django's decorator @cached_property works. Here is the class definition (original source):

# Got from django code base
class cached_property:
    """
    Decorator that converts a method with a single self argument into a
    property cached on the instance.
    Optional ``name`` argument allows you to make cached properties of other
    methods. (e.g.  url = cached_property(get_absolute_url, name='url') )
    """
    def __init__(self, func, name=None):
        self.func = func
        self.__doc__ = getattr(func, '__doc__')
        self.name = name or func.__name__

    def __get__(self, instance, cls=None):
        """
        Call the function and put the return value in instance.__dict__ so that
        subsequent attribute access on the instance returns the cached value
        instead of calling cached_property.__get__().
        """
        if instance is None:
            return self
        res = instance.__dict__[self.name] = self.func(instance)
        return res

This will be used in a class as follows:

class Double():
   def __init__(self, x):
      self.x = x

   @cached_property
   def square(self):
      return self.x * self.x

d = Double(2)
d.square # Prints 4. Performs computation
d.square # Prints 4. Uses the cached value

When the instance d of Double first accesses the square property, the __get__ from the cached_property class is called. This method receives the instance d as one of its parameters. It then calls the actual function and stores the result in the instance's __dict__[square] variable. The next time the square property is accessed, it picks up the appropriate value from its __dict__ and the function is not called. This is because d has a lookup chain starting from b.__dict__['square'] as explained here.

However should the object be changed, for example:

d.x = 3
d.square # Prints 4

the square still prints 4 instead of 9. This is because the square method is still accessing the cached value. To update this, the __dict__[square] parameter has to be cleared before.

d.x = 3
del d.__dict__['square']
d.square # Now prints 9

Clearing this will force a recomputation of the square variable. However, it's really easy to skip this, so I think the cached_property should be used where the objects parameters are not expected to change.

Courage To Be Disliked: A Review

2019-02-13T16:20:00+03:00

Courage to be disliked is a book that explains Adlerian psychology. It does so in the form of a dialog, between a student and a master. Here are the various ideas I got from reading the book:

Etiology vs Teleology

Etiology is when someone focuses on the past, and uses past experiences and events to explain what has currently happened. The text is heavily against this, since it provides an easy way out of various problems that occur in life. The book however emphasizes on teleology, which is where one focuses on the now, and tries to find out why various things have happened. A good example of this is when one goes to the doctor, who doesn't try to figure out what past event caused some disease, but rather focusses on the current symptoms and treating the current ailment.

The biggest problem with etiology however, is that it can lead to self determinism. This is where one believes that how their life progresses has already been determined by events in the past, and that nothing can change this.

Teleology also provides a new insight into various events and reactions. For example, people tend to blame some reactions on their anger, as though it was a separate entity from themselves. However, anger is a tool that can be used to achieve some goal. A good example of this is when a mother is angrily scolding their child. If the mother receives a phone call, she'll put the anger away and calmly receive the call. Afterwards, she becomes angry again and continues to scold the child. This shows that the mother is using anger as a tool against her child.

Teleology also provides a means of understanding why changing one's lifestyle seems difficult. People tend to remain in their lifestyle because it's comfortable and easy to predict. Lifestyles may also provide good explanations for some decisions one has made. For example, someone who dreams of becoming a novelist yet their job keeps them too busy. A teleological explanation of why the novelist doesn't quit would be that s/he is using the job (instead of quiting and focussing on the novel) because he is actually afraid of criticism of his novel or that s/he may produce inferior writing. Later on in their lives, they will use the reason of old age for this too.

Weakness and Strength

Key emotions that tend to help people become better are the feeling of inferiority and the pursuit of superiority. They help support one's striving for growth. However, when these feelings become a hindrance, they develop into complexes. The inferiority complex occurs when one uses their feelings of inferiority as an excuse in life. For example, one can claim that they cannot be successful because they are uneducated, and therefore won't try to overcome this obstacle. The superiority complex is whereby one has the feeling of inferiority but doesn't have the strength to strive for growth. They compensate for this by associating with people or things they feel are superior e.g. some brands of clothes or some celebrities.

Weakness can make one strong and powerful. For example, the baby can control the parents because it is weak. Some people tend to cling to their weaknesses so as to get some hold of people. If someone tries to help them overcome their weakness, they use the fact that the other party cannot possible understand their feelings as a deterrent. However, in doing so, they become slaves to their weaknesses and end up unable to survive without this.

Going through life as though it is a competition is another form of weakness. This is because if competition is at the core of one's interpersonal relationships, one will start viewing people as enemies who must not be underestimated. This makes the world a scary place to live in.

Also when having a discussion with someone, the moment it becomes a power struggle, one should step down. This is because the conversation has derailed from that of providing information/opinion to that of winner and losers. Also, avoid criticizing the other party just because you think you're right, otherwise the conversation will degenerate into a power struggle.

Life Tasks

Another interesting concept introduced in the book is that of tasks. An individual has three main classifications of tasks:

Task of friendship
Task of work
Task of love

Separation of tasks is whereby one decides to what point one's own tasks go, and from which point the tasks become another person's. Once this is done, one should do their own tasks to the best of their ability, and only lend a hand to others when needed. However, one should be careful not to encroach on another person's territory. To help with this, one needs a good way of figuring out ownership of a task. An indication of this is who will eventually benefit if the task is done.

In interpersonal relationships, it is the individual's task to believe in the other party. How the other party behaves with regard to said belief is not the individual's task. So if the other party takes advantage of you, that is still their task. If one feels the relationship is becoming problematic, it should be severed (which is the individual's task).

It is also one's task to live one's life the best way one believe in. The judgement passed by people about one's life is not one's task. This therefore means one is free to do as one pleases. In fact, freedom can be defined as being disliked by other people. This is because being disliked by others is proof that one is living life as one would please, not trying to satisfy others.

Furthermore, if a task is beyond one's capability, one needs to accept this and move forward in doing what one can do (self acceptance) thus one avoids lying to oneself. This is counter to self affirmation, where one uses phrases like 'I can do it', when something is cleary beyond one's ability. A good example for this is in exams, where if someone fails they will either say 'I was unlucky this time' (self affirmation) or accept what they got and try to improve for the next exam (self acceptance).

People try to be special because they cannot accept their normal selves. They therefore equate being normal with being incapabale. However, with self acceptance, once accepts their ordinary selves. One should then have the courage to be normal.

The problem with aiming to be special (having lofty goals) is that one views life like climbing a mountain, where if they do not reach the peak, they have not reached their "real life". This means they are not living in the present. The greatest life lie of all is in fact to not live here and now. A better alternative is to view life as a series of moments, without a destination. This way, one ends up living their "real life" every day, every hour and every minute.

"Life in general has no meaning. Whatever meaning life has must be assigned to it by the individual."

Sway Window Manager

2019-01-12T08:00:00+03:00

Wayland is a display server that should work as a replacement for X. I've been thinking of trying it out for some time now. Detailed information about wayland and its history can be found in this article. The differences between xorg and wayland can be found here or in this thread. It seems to me the biggest problem with wayland is that it is not network aware (although I haven't used this feature yet in xorg preferring using vnc servers instead) and that some programs will not work with it (for example, redshift).

However, the biggest block for me was that my i3 workflow had been ingrained in my muscle memory and I did not want to lose this. Luckily, I found sway. Surprisingly, it worked with most of my i3 configuration, only having to make changes to custom mapping applications that were x based e.g. feh.

To set sway up, I decided to avoid the version used in pacman, because it seemed to be too old. I went with bleeding edge sway and set it up using:

yay -S wlroots-git sway-git

With this I get the latest version plus all its improvements. The problem though, is that sway sometimes fails to start after a system update. To fix this, I just start up i3 and run the above command again.

This might be subjective, but I found the colours in swaywm when I first used it to be better and brighter. This is something I had always wanted to figure out how to fix in i3, but never got around to doing this.

I also felt that out of the box, it had better support for my touchpad. It did not have a lot of ghost touches, random scrolling behaviour, etc. which in i3, I have to spend a lot of time tweaking and configuring the touchpad settings. In sway, these are the only configurations I had to enable:

input "1267:12328:ELAN1300:00_04F3:3028_Touchpad" {
    dwt enabled
    tap enabled
}

The problems I have with sway are:

I haven't found a way to set up a floating terminal window. I have defaulted to using a floating window usint this configuration:
```
bindsym $mod+F12 exec xfce4-terminal --drop-down --title=dropdown
for_window [title="dropdown"] floating enable
```
Switching between keyboard layouts is tricky. The options I've got involve restarting sway with the new keyboard layout I intend to use, for example:
```
export XKB_DEFAULT_LAYOUT=us; export XKB_DEFAULT_VARIANT=qwerty; sway
```
When connecting an external monitor to my laptop, I expect it to be on the right of the laptop screen. However, sway defaults with it on the left. To fix this, I set up both screens and their index as shown below:
```
output eDP-1 resolution 1920x1080 position 0,0
output HDMI-A-1 resolution 1280x1024 position 1920,0
```

Summary of 2018

2018-12-31T19:00:00+03:00

This year has been meh for me ;( The highlights for the year include:

Completing course work for my masters class
Starting research on Renewable Energy Systems and how best to size them and control this system.
Active participation in various communities I managed to join especially Urban Perspective and NaiLUG.
Achieved most of the fitness goals I had intended to achieve at the start of the year. Key amongst these is jogging at least 3 times a week.
I also started doing some investments this year, mostly passive. Part of my investment strategy had been in agribusiness (active invesetment) this year, but things did not go too well of that front. At least I achieved some level of financial literacy this year, and have a better strategy for the oncoming year.
I attended my first _ruracio_ this year. It was a unique experience merging traditional culture with modern expectations.
I had my first trip outside Nairobi. I managed to visit Naivasha and Elementaita on an unofficial class trip. I realised I like to travel (hopefully this will remain true) and will endeavour to visit more places next year.
Managed to sort out internet at home. This by far had the most impact on my life this year. Having to use the internet without limits (as had been initially the case with the bundles options I was having with Telkom) was really refreshing. I could now experiment more, play with more technologies and not have to think about effects of data on my wallet.

I also managed to read some really interesting books this year, like Deep Work and Lauren Ipsum. I didn't get to my goal of 20 books and I'm blaming this on my loose concentration. This is whereby I'll leave a half finished book if and when I get something more interesting. I hope to improve on this next year.

Fixing Small Fonts on My Laptop

2018-09-17T16:20:00+03:00

I have been having this problem with my laptop where the fonts seem too small. To fix this, I first had to check out my laptop's screen configurations:

xrandr

Part of the output of this command is shown below. As can be seen it shows that my laptop screen is 'eDP1' and has dimensions of 290mm by 170mm.

Screen 0: minimum 8 x 8, current 1920 x 1080, maximum 32767 x 32767
eDP1 connected primary 1920x1080+0+0 (normal left inverted right x axis y axis) 290mm x 170mm
   1920x1080     60.05*+  59.93
   1680x1050     59.88

Another command that can find screen information is:

xdpyinfo| grep -B2 resolution

This command shows different screen dimensions. Its output is shown below:

screen #0:
  dimensions:    1920x1080 pixels (508x285 millimeters)
  resolution:    96x96 dots per inch

There is some discrepancy somewhere. To fix this, I had to create a monitor configuration file in /etc/X11/xorg.conf.d/90.monitor.conf that contained the correct screen dimensions. I got the screen dimensions by measuring them with a tape measure. The contents of the file are:

Section "Monitor"
    Identifier             "eDP1"
    DisplaySize            290 170
EndSection

I then restarted i3 and my screen read comfortably. The output from the xdpyinfo command shows:

screen #0:
  dimensions:    1920x1080 pixels (290x170 millimeters)
  resolution:    168x161 dots per inch

I got most of the information to fix this here.

Bluetooth Speakers Problems

2018-09-14T08:00:00+03:00

Connecting to Device

To connect to the bluetooth speakers, I use the bluetoothctl commandline options. The following are commands I run for this to work:

sudo systemctl start bluetooth.service
bluetoothctl
# Within the bluetoothctl environment
power on
agent on
default-agent
scan on
pair A4:A7:B5:22:E6:F5
connect A4:A7:B5:22:E6:F5

If the device had been previously paired, I skip the pair step.

Sometimes, my bluetooth speakers would fail to connect. To fix this, I usually remove the item from the list of paired devices and try to pair again before connecting.

remove A4:A7:B5:22:E6:F5

Another problem I've experience is whereby the device connects but I can't hear sound from the speakers. To fix this, I first confirm that the device is set up as an a2dp device in pavucontrol. If this change has been made and there is still no sound, kill pulseaudio.

pulseaudio --kill

Audacity

Audacity could not link up with my bluetooth speakers during playback. However, the bluetooth speakers were visible from pavucontrol and I could hear audio from them when using other applications. To fix this, I had to install alsa-plugins.

sudo pacman -S alsa-plugins

Once this was done, the options pulse were available in audio recording and play back devices.

Freeing up Space in my Linux Machine

2018-08-26T16:00:00+03:00

I have a 256GB SSD on my laptop. This is rather constrained and I occasionally end up with less than 10GB drive space. When this happens, I have to figure out what is taking up all my space and fix this. This prompts some form of search and cleaning up of the culprits. This article lists the process and commands I usually run for this to happen.

To get a general feel of how much space I have:

df -h

Some sample output from this command is:

Filesystem      Size  Used Avail Use% Mounted on
dev             3.9G     0  3.9G   0% /dev
run             3.9G  1.4M  3.9G   1% /run
/dev/sda5       106G   93G  7.4G  93% /
tmpfs           3.9G  132M  3.8G   4% /dev/shm
tmpfs           3.9G     0  3.9G   0% /sys/fs/cgroup
tmpfs           3.9G  8.0K  3.9G   1% /tmp
/dev/sda6        41G   20G   21G  50% /run/media/silo
tmpfs           787M   20K  787M   1% /run/user/1000

This provides key information like what is the size of a partition and how much space has been used up. The '-h' tag helps in making the sizes easier to read.

To list the various folders in root folder '/' and their sizes, I use:

du -sch /* | sort -h

If permissions become a problem, you can run it with sudo.

11G     /var
14G     /opt
20G     /run
33G     /home
36G     /usr
113G    total

The above shows part of the output of the above command. You can change the folders as you try to figure out what is consuming space.

The du command ignores files and folders that start with a '.'. This can become troublesome because it will not give you enough information about what is consuming space especially in the home directory, which in my case is riddled with such. To fix that, run this command instead:

du -sch .[\!.]* * | sort -h

This will scan all folders including those that start with a '.'. However it returns results of the current folder one is in. A more generic one is:

du -sch /folder/path/[\!.]* * | sort -h

The ls command is another alternative for finding the sizes of files:

ls -lah

Using the above commands, a general idea of where and how disk space is used can be formed.

Cleaning Disk Space

Clearing up the cache of whatever installer I'm using is usually a good place to start:

sudo pacman -Scc
sudo yay -Scc

Some installed packages take up a log of space. This command (found here) lists packages from the smallest to the largest.

pacman -Qi | grep 'Name\|Size\|Description' | cut -d: -f2 | paste - - - | awk -F'\t' 'BEGIN{ s["MiB"]=1024; s["KiB"]=1;} {split($3, a, " "); print a[1] * s[a[2]], "KiB", $1}' | sort -n

You can also remove orphans. To find them:

sudo pacman -Qtd

Look through the list and remove what is not needed. To remove all orphans:

sudo pacman -Rns $(pacman -Qtdq)

Delete unnecessary files that are hogging up space. These can be found using the dh or ls commands and deleted using:

rm file
rm -r directory

Another huge disk space hog is my cache folder. I typically delete this folder using:

rm -r ./cache/*

But if you want to be safe, you can delete only the files in that folder that haven't been used in a year using:

find ~/.cache/ -type f -atime +365 -delete

I use docker a lot. So it tends to eat up a lot of disk space. The following commands help me out here:

sudo docker system df -v
sudo docker system prune
sudo docker volume prune

Without the '-v' option, the command offers a summary of the disk space used by docker.

To deal with containers, I use the following commands:

sudo docker ps -a
sudo docker rm NAME

The first command will list all containers and the second command deletes containers based on their name.

To deal with images, I first list all the images with:

sudo docker images -a

I usually like deleting the images with no tags. To that I run the folloing command:

sudo docker images | grep \<none\> | awk '{print $3}' | xargs sudo docker rmi

With this I can effectively remove files until I feel comfortable with the remaining disk space.

2018 So Far

2018-08-03T19:00:00+03:00

This year so far has been tough.

I thought I would have read around 10 books by now but I'm barely half that mark. Most of the books are mostly fiction, but at least I've got some variety this time. The books I've completed so far are:

'The Worthing Saga' by Orson Scott Card: This is a really interesting read, although I disagree with what I think is the core message of this book. This is that someone has to suffer so that they can enjoy life fully, otherwise their lives are empty.
'Deep Work' by Cal Newport: This has to be the best book I've read this year. It will be even better if I can manage to put the advice in this book into practice. It explains the concept of concentrating on one's work and the advantages it adds to one's life. This concept comes with various examples of people that have applied deep work in their practice to successful extents.
'The Princess Bride' by William Goldman: The feel-good-book of the year. The story just flows and its told in a weird way where the author occassionally adds commentary to provide context or sometimes give spoilers in the book.
'Lauren Ipsum' by Carlos Bueno: This book is aimed at kids but is really awesome. Its various puns and analogies to computers were really cool.
'The Great Hunt (Wheel of Time Book 2)' by Robert Jordan: This series does get really interesting in book 2. The first book was a bit dull but this picks up the pace to tell a great tale of the Dragon.

Some books I've started reading but yet to complete are:

'Professor Frisby's Mostly Adequate Guide to Functional Programming' by Brian Lonsdorf: This explains functional programming concepts using javascript. This has to be the book that got me interested in functional programming, especially when the author explained currying and composing. That was literally mind-blowing.
'LearnYouAHaskell' by Miran Lipovaka: Decided to learn a purely functional language after getting through some chapters of Mostly Adequate. I'm barely done with the third chapter and I'm reall liking this way of thinking.

I haven't blogged as much as I thought I would this year. Basically the idea was to blog on what I was doing so that it acts a means of documentation for my future self. I find it easier to read my own words especially when its from a concept I had forgotten about. The good thing is I have a lot of new things I've learned, so I really need to pull up my pants. Some topics I've really got into include:

Robotics: I'm thinking of making a javascript tool that will do the forward kinematics, inverse kinematics, robotic dynamics (Euler-Lagrange and Newton-Euler) and motion planning. I think doing this will cement the knowledge I've got a bit more.
Functional programming: Not sure of what to do, but I think I'll look for a pet project that I can implement using haskell.
Model Predictive Control and Advanced Control: These are some really dope units I'm doing in my Master's program. Its mostly theoretical, but I have to find something practical to use these in. I mean, just thinking about these concepts puts a smile to my face.
Hybrid Energy Systems: This is what I want to focus on for my Masters' thesis though I'm still looking for a gap here.

For the comic site, I just realized my art isn't that good. I think this has to do with self-bias, where immediately after drawing something, I looked at it and saw that it was good. But months later (when the feeling of achievement has waned), I look at them objectively and I'm shocked that I found them good. I've been looking at various videos and trying to practice. This is something I intend to become better at.

I'm also in my second semester for masters in Mechatronics Engineering. The course work is pretty interesting but difficult. The killer feature though is the research work, more specifically the research proposal. This becomes more impossible with every passing day especially finding a viable gap. I had some things I really wanted to get into, so I started digging in and reading papers. I ended up (still do) frustrated and angry at myself because I mostly couldn't understand these papers. And when I finally get a paper I somehow understand, I can't find any gap to fill. This has been going on for the past three months and its maddenning. I've thought that there must be a better way to do this, but this is the process my institution takes. I get my own gap, and write a proposal around it (with barely any feedback or help). So right now I'm just shooting in the dark and hoping something comes up.

For personal projects, Mradi is coming up nicely. Development has slowed down though in recent months. Hopefully I will manage to finish this product soon. I also started working on some financial tool called Staafu. It should basically help one plan for retirement by maximizing investments and reducing risks.

Document Scaning in Arch Linux

2018-07-08T15:00:00+03:00

Using a scanner in arch is pretty perplexing at first, especially if you want to try it from the command line. Here are my steps to end up with something that works.

First install the sane package. Sane provides a command line tool for using scanners.

sudo pacman -S sane

If the scanner is a hp brand, setting up hp-lip also helps out.

sudo pacman -S hplip

This gives features that help setting up the printer or scanner easily like hp-setup. Using it and the various prompts, one can set up the printer and scanner easily.

Assuming the device has already been set up, find a list of devices using:

scanimage -L

My output looks like this:

device `v4l:/dev/video0' is a Noname USB2.0 HD UVC WebCam: USB2.0 HD virtual device
device `hpaio:/net/HP_LaserJet_Pro_MFP_M127fw?ip=192.168.1.27' is a Hewlett-Packard HP_LaserJet_Pro_MFP_M127fw all-in-one

To determine options for my device I have to do:

scanimage --device "device name" -A

Which in my case is:

scanimage --device 'hpaio:/net/HP_LaserJet_Pro_MFP_M127fw?ip=192.168.1.27' -A

This in my case outputs:

All options specific to device `hpaio:/net/HP_LaserJet_Pro_MFP_M127fw?ip=192.168.1.27':
  Scan mode:
    --mode Lineart|Gray|Color [Lineart]
        Selects the scan mode (e.g., lineart, monochrome, or color).
    --resolution 75|100|150|200|300|600|1200dpi [75]
        Sets the resolution of the scanned image.
    --source Flatbed|ADF [Flatbed]
        Selects the scan source (such as a document-feeder).
  Advanced:
    --brightness -1000..1000 [0]
        Controls the brightness of the acquired image.
    --contrast -1000..1000 [0]
        Controls the contrast of the acquired image.
    --compression None|JPEG [JPEG]
        Selects the scanner compression method for faster scans, possibly at
        the expense of image quality.
    --jpeg-quality 0..100 [inactive]
        Sets the scanner JPEG compression factor. Larger numbers mean better
        compression, and smaller numbers mean better image quality.
  Geometry:
    -l 0..215.9mm [0]
        Top-left x position of scan area.
    -t 0..296.926mm [0]
        Top-left y position of scan area.
    -x 0..215.9mm [215.9]
        Width of scan-area.
    -y 0..296.926mm [296.926]
        Height of scan-area.

And using this information I can scan the document. For example to output to a tiff file, I'll do:

scanimage --device 'hpaio:/net/HP_LaserJet_Pro_MFP_M127fw?ip=192.168.1.27' --format=tiff --mode=color  --resolution=75 > 24.tiff

This sets the mode to color and the resolution to 75. These options are found from the previous output.

To get the output into a pdf file, I use tiff2pdf. If I have multiple files that I scanned, I add them into a single file using tiffcp.

tiffcp 1.tiff 2.tiff 3.tiff output.tiff
tiffpdf -j output.tiff -o output.pdf

A more advanced example is:

tiff2pdf -p letter -j -q 75 -t "title" -f -o title.pdf doc.tiff

However using tiff2pdf occasionally fails because libtiff has set and upper limit of memory to be used by libjpeg. So if I have really large tiff files, I'll most likely get the following error:

JPEGLib: Backing store not supported.
tiff2pdf: Error writing encoded strip to output PDF sm.pdf.
tiff2pdf: An error occurred creating output PDF file.

I could not find a fix for this. So I just made sure my tiff files were less than 10MB. If a file became more than that size, I reduced the resolution of scanning. Also while combining files using tiffcp, if the eventual tiff file is too big, I convert the individual tiff files into pdfs and combine the pdfs using the following command:

pdfunite 1.pdf 2.pdf output.pdf

And that is how I scan document in my arch setup.

When Memory is not enough

2018-05-31T16:00:00+03:00

A computer or server suddenly freezes or crashes. You don't know what the problem is and you really need to complete this task. One of the options is to consider memory. You might have just run out. To confirm this, you can monitor memory usage using the htop command while the operation is running. This has occured to me multiple times, most recently while installing tor using yaourt, compiling a nodejs project and compiling hledger.

To fix this, one needs to increase the size of swap. Swap is the portion of the hard disk that the OS will use when memory runs out, like some form of virtual memory. It is usually existent on most linux installations (You probably set one yourself), but you can increase this to cater for this unique situation.

So to do this, first allocate some diskspace that will be used for this task. This saves us this space and it cannot be used for any other task. Afterwards change the ownership and permissions of the allocated space, then set it up as a linux swap space and finally enable it so that the OS can use it for swapping..

sudo fallocate -l 10G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
swapon /swapfile

Then, you can run whatever task was failing/crashing. After the task successfully runs, you might want to disable this new space. To do that:

swapoff -a
rm -f /swapfile

To improve performance, one can try to experiment with the virtual memory subsystem. More on this can be found in the documentation for vm. For example, I tried this:

sudo sysctl vm.swappiness=10
sudo sysctl vm.vfs_cache_pressure=50

To make the changes permanent, one can try this:

sudo cp /etc/fstab /etc/fstab.bak #create backup just in case something fails
echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab
echo 'vm.swappiness=10' | sudo tee -a /etc/sysctl.conf
echo 'vm.vfs_cache_pressure=50' | sudo tee -a /etc/sysctl.conf

The source for these commands is from the archlinux_wiki

Jogging Time Tracking In Google Sheets

2018-03-31T19:00:00+03:00

I jog frequently. I needed to track the various times I spend jogging my various routes. There are pretty awesome android apps for this but carrying a phone with me while jogging did not work. I was ok with a wrist watch though. So I jog and afterwards store the time and route taken in google sheets. A sample table would look like this:

A	B	C
Date	Route	Time
2018-01-04	Route A	21.40.60
2018-01-05	Route A	21.78.80

It becomes hard to gain insight into this table if it becomes too long. So I needed a method of getting basic metrics from this data. This was the average times I spend on a route and the best time recorded on a route.

I could not find a function I could easily use to get this information. This was because I store my times in a weird format, that is Minutes.Seconds.Centiseconds. I used this because it is what my wrist watch gave me. So I had to write my own custom functions.

Before doing calculations in time, it is best to convert it all to the smallest unit you have (in my case it was centiseconds). This is the function I came up with:

function getTimeInMicroSeconds(timeGiven) {
  var time = String(timeGiven).split('.');
  return parseInt(time[0])*6000 + parseInt(time[1])*100 + parseInt(time[2]);
}

So to get the average of a group of times, you just add up the times in milliseconds, then find the average in milliseconds, and convert the average back to minutes, seconds and hours. To do that:

  var total_time = 0;
  var total = input.length;

  for (var i =0 ; i<total; i++){
    var time = getTimeInMicroSeconds(input[i]);
    total_time += time;
  }

  var average_time = Math.floor(total_time/total);
  var microseconds = average_time % 100;
  var seconds = Math.floor(average_time/100) % 60;
  var minutes = Math.floor(average_time/60000);
  return correctSizeString(minutes,2)+"."+correctSizeString(seconds,2)+"."+correctSizeString(microseconds,2);
}

The function correctSizeString just ensures the times are appropriately sized e.g. instead of showing 6 in seconds time, it will show 06.

To find the best time, I came up with this function:

function BESTTIME(input) {
  if (!input.map)
    return input;

  minimum_time = Infinity;
  index_minimum = -1;

  for (var i=0; i<input.length; i++) {
    var time = getTimeInMicroSeconds(input[i]);
    if (time < minimum_time){
      minimum_time = time;
      index_minimum = i;
    }
  }
  return input[index_minimum];
}

And this is one of the functions I use to get average time:

=AVERAGETIME(FILTER($C$2:$C,$B$2:$B=$E2))

The function filters the input based on what is in row B (which is the route) and finds the average of the times of a particular route.

The complete script can be viewed from this gist.

Testing Asynchronous Calls in VueJS

2018-03-16T15:00:00+03:00

I frequently use asynchronous calls when coding in vuejs, especially when I get and send data to an online api. Initially, I found it difficult to test the asynchronous calls, but as I learned more, asynchronous tests became painless. I detail the methods I use to test promises and async calls in this article. The snippets here assume you are using axios for http requests and the vuejs project uses vuex for state management.

This is a simple vuex store that can get a random quote or a list of quotes from this online api.

import Vue from 'vue'
import Vuex from 'vuex'

Vue.use(Vuex)

export default new Vuex.Store({

  state: {
    quotes: [],
    randomQuote: null
  },
  mutations: {
    setQuotes (state, newQuotes) {
      state.quotes = newQuotes
    },
    setRandomQuote (state, randomQuote) {
      state.randomQuote = randomQuote
      console.log(state.randomQuote)
    }
  },
  actions: {
    getQuotes ({commit}) {
      return Vue.axios.get('http://quotesondesign.com/wp-json/posts')
        .then((response) => {
          commit('setQuotes', response.data)
        })
    },
    getRandomQuote ({ commit }) {
      return Vue.axios.get('http://quotesondesign.com/wp-json/posts?filter[orderby]=rand&filter[posts_per_page]=1')
        .then((response) => {
          commit('setRandomQuote', response.data)
        })
    }
  }
})

This is vue component that will display the list of random quotes got:

<template>
  <div>
    <h2>Random Quote</h2>
    <div id='random-quote' v-for="item in randomQuote" :key="item.ID">
      <h2>{{ item.title }}</h2>
      {{ item.content }}
    </div>
    <div v-if="randomQuote === null">It is null</div>
    <button id='get-random' v-on:click="getQuote">GetRandomQuote</button>
  </div>
</template>
<script>
import { mapState } from 'vuex'

export default {
  name: 'GetQuotes',
  computed: mapState({
    randomQuote: state => state.randomQuote
  }),
  methods: {
    getQuote: function () {
      this.$store.dispatch('getRandomQuote')
    }
  }
}
</script>

If I want to test the getQuote method, I'll have to figure out how to make the asyc ajax call to the server. Since the call happens in another thread, it becomes tricky, because I have to wait for that thread to finish before testing the result. Luckily, there is a library called moxios. To set it up on the project, do:

npm install moxios --save-dev

With moxios, I can mock the ajax calls (from axios). With this, I get dummy content that the test can treat similarly to how the component will treat server results. Moxios also has a wait function, which will pause execution of test until the promise completes. To test the above component with moxios:

import Vue from 'vue'
import Vuex from 'vuex'
import { mount } from 'vue-test-utils'
import RandomQuote from '@/components/RandomQuote'
import store from '@/store'
import axios from 'axios'
import VueAxios from 'vue-axios'
import moxios from 'moxios'

Vue.use(Vuex)
Vue.use(VueAxios, axios)

describe('Login.vue', () => {
  let wrapper

  beforeEach(() => {
    moxios.install()

    moxios.stubRequest('http://quotesondesign.com/wp-json/posts?filter[orderby]=rand&filter[posts_per_page]=1', {
      status: 200,
      response: [
        {
          ID: 1622,
          title: 'Stephen Anderson',
          content: '<p>At this point in experience design evolution, satisfaction ought to be the norm, and delight out to be the goal.</p>\n'
        }
      ]
    })

    wrapper = mount(RandomQuote, {
      store
    })
  })

  afterEach(() => {
    moxios.uninstall()
  })

  it('should show random quote when clicked', (done) => {
    wrapper.find('#get-random').trigger('click')
    moxios.wait(() => {
      let randomQuote = wrapper.find('#random-quote')
      expect(randomQuote.text()).toContain('At this point in experience design evolution, satisfaction ought to be the norm, and delight out to be the goal.')
      done()
    })
  })
})

From the above, I use stubRequest to mock the http request. In the actual test, I use moxios.wait(), which will wait until the api call completes before testing the contents of the random quote. I like this method because it means I get to test the store indirectly while testing my components.

Instead of moxios.call(), I could use vue's nextTick(), which also waits for the promise to complete.

it('should show random quote when clicked', (done) => {
  wrapper.find('#get-random').trigger('click')
  wrapper.vm.$nextTick(() => {
    let randomQuote = wrapper.find('#random-quote')
    expect(randomQuote.text()).toContain('At this point in experience design evolution, satisfaction ought to be the norm, and delight out to be the goal.')
    done()
  })
})

Another cleaner method of testing promises is using the library flush-promises. With flush-promises, the test would look like:

it('should show random quote when clicked', async () => {
  wrapper.find('#get-random').trigger('click')
  await flushPromises()
  let randomQuote = wrapper.find('#random-quote')
  expect(randomQuote.text()).toContain('At this point in experience design evolution, satisfaction ought to be the norm, and delight out to be the goal.')
})

The advantage flushPromises has is that if the component has multiple asynchronous calls at the same time e.g. call to user api to confirm if logged in, get list of quotes, get comments, and for some reason the test does not work, we can just add multiple flushPromises in the test.

it('should show random quote when clicked', async () => {
  wrapper.find('#get-random').trigger('click')
  await flushPromises()
  await flushPromises()
  let randomQuote = wrapper.find('#random-quote')
  expect(randomQuote.text()).toContain('At this point in experience design evolution, satisfaction ought to be the norm, and delight out to be the goal.')
})

Jupyter Notebook Setup on a VPS

2018-03-02T15:00:00+03:00

These steps have been tested on Digital Ocean server and a scaleway ARM64 server. I used the cheapest provided options for my setups.

First, set up ubuntu on the server. Both Digital Ocean and Scaleway provide an easy way to do this. Just follow their prompts, and you'll have your server up and running. After the server is ready, update ubuntu:

sudo apt update
sudo apt upgrade

The server only has the root user. It is not advisable to use root for normal work. So, I create a new user and switch to it:

useradd -m -s /bin/bash user
usermod -aG sudo username
sudo passwd username
su - user

Pip will be used to install various python packages. To install it:

sudo apt install python3-pip

I then install various ml libraries I'll use later:

python3 -m pip install --user numpy scipy matplotlib pandas sympy nose seaborn pillow keras tensorflow scikit-learn scikit-image kaggle-cli
pythom3 -m pip install ipython[notebook] jupyter

When setting up scaleway, some of the libraries failed to install. Here are the dependenices I had to install for pillow:

sudo apt-get install libtiff5-dev libjpeg8-dev zlib1g-dev libfreetype6-dev liblcms2-dev libwebp-dev tcl8.6-dev tk8.6-dev python-tk

And here are the dependencies I had to install for scikit:

sudo apt-get install gcc gfortran python-dev libopenblas-dev liblapack-dev

And for kaggle-cli too, I had to do this:

sudo apt-get install libxml2-dev libxslt1-dev python3-lxml

Tensorflow cannot be installed in the arm64 scaleway instance using pip. So remove it from the packages.

The packages are installed, but cannot be got from the terminal. This is because the bash environment does not have it in its PATH variable. To fix this, add this to the ~/.profile file:

if [ -d "$HOME/.local/bin" ] ; then
        PATH="$HOME/.local/bin:$PATH"
fi

Having set up most of the libraries I'll need, I have to configure jupyter so that I can access it from my browser.

jupyter notebook --generate-config

A file is generated in ~/.jupyter/jupyter_notebook_config.py. This file contains various configuration options for jupyter.

I change the following:

c.NotebookApp.open_browser = False
c.NotebookApp.ip = '0.0.0.0'

To set up the password to be used on logging in do:

jupyter notebook password

I then start jupyter using:

nohup jupyter notebook

The jupyter can be accessed from : ip_address:8888 using any browser.

To install Tensorflow on scaleway's arm processors, I found instructions for a custom build here.

The instructions are to:

curl -L https://github.com/lherman-cs/tensorflow-aarch64/releases/download/r1.4/tensorflow-1.4.0rc0-cp35-cp35m-linux_aarch64.whl > /tmp/tensorflow-1.4.0rc0-cp35-cp35m-linux_aarch64.whl
python3 -m pip install /tmp/tensorflow-1.4.0rc0-cp35-cp35m-linux_aarch64.whl

Control: Controllability and Observability

2018-02-21T15:00:00+03:00

The state space equations are given by:

\begin{align*} x'(t) = Ax(t) + Bu(t)\\ y(t) = Cx(t) + Du(t) \end{align*}

If we discretize the system, the equation can be written as:

\begin{align*} x(n+1) = A_dx(n) + B_du(n)\\ y(n) = C_dx(n) + D_du(n) \end{align*}

The discretization steps can be found here:discretization

Controllability

A system is said to be controllable if the state can be changed by changing the inputs.

Using the discretized equation, we can come up with the following:

\begin{align*} x(1) &= A_dx(0) + B_du(0)\\ x(2) &= A_dx(1) + B_du(1) = A_d^2 x(0) + A_dB_du(0) + B_du(1)\\ .\\ x(n) &= A_d^n x(0) + A_d^{n-1}B_du(0) + A_d^{n-2}B_du(1) + ... + B_du(n-1) \end{align*}

The last equation can be written as:

\begin{equation*} x(n) - A_d^n x(0)= \begin{bmatrix} B_d & A_dB_d & A_d^2B_d & ... & A_d^{n-2}B_d & A_d^{n-1}B_d \end{bmatrix} * \begin{bmatrix} u(n-1) \\ . \\ . \\ . \\u(2) \\ u(1) \\ u(0) \end{bmatrix} \end{equation*}

Replacing with the controllability matrix with C, we get:

\begin{align*} x(n) - A_d^n x(0) = C * U \\ \therefore\\ U = C^{-1} * (x(n) - A_d^n x(0)) \end{align*}

This means that the equation is solvable when $C^{-1}$ exists, i.e. its a non singular matrix.

Therefore a system is controllable when:

\begin{equation*} \begin{bmatrix} B_d & A_dB_d & A_d^2B_d & ... & A_d^{n-2}B_d & A_d^{n-1}B_d \end{bmatrix} \end{equation*}

has a determinant.

Observability

Using the discretized equation too:

\begin{align*} y(0) = C_dx(0)\\ y(1) = C_dx(1) = C_dA_dx(0)\\ y(2) = C_dx(2) = C_dA_d^2x(0) \\ y(n) = C_dx(n) = C_dA_d^nx(0) \end{align*}

This can be written as:

\begin{equation*} \begin{bmatrix} y(0)\\y(1)\\y(2)\\.\\y(n) \end{bmatrix} = \begin{bmatrix} C_d \\ C_dA_d \\ C_dA_d^2 \\ . \\C_dA_d^n \end{bmatrix} * x(0) \end{equation*}

The observability matrix is:

\begin{equation*} \begin{bmatrix} C_d \\ C_dA_d \\ C_dA_d^2 \\ . \\C_dA_d^n \end{bmatrix} \end{equation*}

And the systems has a unique solution if the system has a rank n. If the system is a square matrix, the solution is of rank n if the determinant is not 0.

The observability and controllability derivations can be found from here:rutgers

SSH AND SERVERS

2018-02-16T15:00:00+03:00

SSH provides a secure means of remote login from one computer to another computer. When setting up linux servers, the OS usually provides ssh by default. To login:

ssh user@ipaddress

This prompts for a password, after which you have access to a remote terminal. Sometimes, no password is requested, and you are required to set one up after your first login.

The password login method is not really secure. This is because someone can try to guess it. A more secure method is to disable the password login and set up ssh-keys on the server. Therefore, someone will need to have the private ssh-key to be able to access your server.

To generate a key:

ssh-keygen -t rsa -b 4096 -C 'comment'

There will be some prompts that guide you through the process. If you choose to save the key in the default location, this key will be used wheneve a custom key is not provided. You should take care however, since you can easily overwrite a key that was in use. The command generates two files, the private key and a public key ( has a .pub suffix).

To set up the key, copy the public key to the server by:

ssh-copy-id -i file.pub user@ipaddress

If you are copying the keys found in the default location i.e. ~/.ssh/id_rsa.pub, the -i option can be ignored.

Having set the key up on the server, you can log in using:

ssh -i /path/to/privatekey user@ipaddress

If using the default file location, the -i option can be ignored.

To disable the password login option, log into the server and edit the /etc/ssh/sshd_config file (to disable password access for the server) or ~/.ssh/config file (to disable password access for a specific user). The following line should be added to the file:

PasswordAuthentication no

After this, restart ssh using:

reload ssh

SSH Forwarding

SSH forwarding is where the private key used to login to the server, can be used by the server to perform some tasks. This is useful in CI/CD, or when working with git on a server(Need to push to a repository on github).

To enable this, edit the ~/.ssh/config file from the local machine and add:

Host example.com
    ForwardAgent yes

To set up the key for use by the server:

eval $(ssh-agent)
ssh-add ~/.ssh/key

Eval sets environment variables to ouput of ssh-agent. To confirm the key has been added, you can do:

ssh-add -L

When you connect to the server example.com, it will now have you private sshkey in its environment and can be used to access repositories, other servers, etc.

Frozen ssh session

Sometimes after connecting to the server, you might lose your internet connection or hibernate/sleep you computer. The ssh prompt after this will be frozen, with no commands running. To fix this, you have to leave the prompt and log in again to the server. To do this, press the following keys in order from the prompt:

Enter
~
.

System Clock Prescalers and Their Hazards

2018-02-04T15:00:00+03:00

One means of saving power with the attiny13A is by slowing down the clock frequency of the microcontroller. One option of doing this is by using the system clock prescaler. The attiny13A is shipped with running on a 9.6MHz internal RC Oscillator. It has the CKDIV8 fuse set (which provides a system clock prescaler of 8), thus it runs on a frequency of 1.2MHz.

Supposing we want to slow it further, we can set up the CLKPR (Clock Prescale Register) bits for the same. If we set this to a prescaler of 16 for example, the attiny will run at a frequency of 600KHz. However, there is a special write procedure provided for this to prevent unintentional changes:

Write the Clock Prescaler Change Enable (CLKPCE) bit to one and all other bits in CLKPR to zero.
Within four cycles, write the desired value to CLKPS while writing a zero to CLKPCE.

To follow this, our code will be:

// set CLKPCE to 1 and other bits to zero
CLKPR = 0b10000000;
// set prescaler to 256
CLKPR = 0b00001000;

Furthermore, when compiling we have to use the optimization flags -0s, otherwise the process might take more than four cycles. The microcontroller will be running at a clock speed of 37.5KHz after the operation.

Hazards

If successful, however, programming the microcontroller will be difficult. This is because of syncing issues, whereby the microcontroller is significantly slower than the programmer. To fix this you can set the -B flag using avrdude. However, the usbasp programmer I am using does not have this functionality.

To fix this we have to analyze how the microcontroller sets the new frequency. First, the microcontroller will boot up. At this state CLKPR is still its default value. Then the two instructions setting the prescaler are run. This therefore means that if we can find a way to stop those two instructions from running, we can program the microcontroller with another firmware.

IF we activate the RESET pin, before the clock prescaler is set, the chip can be reprogrammed. Connect Ground signal to the RESET pin and then provide power to the programmer (usbasp). This maintains the microcontroller in Reset mode, thus the instructions changing the prescaler are not carried out. Then immediately flash the attiny. If successful, the error will be gone.

Another way to prevent this is to have a considerable delay before the CLKPCE bits are set. This delay allows for reprogramming if necessary.

SciKit-Learn GraphSearch CV

2018-01-24T19:00:00+03:00

Scikit learn provides a lot of algorithms to help in machine learning. However, sometimes those algorithms have too many options. It is not usually evident which options will produce the best results. One method of solving this is to loop through all possible options and check the score.

crit = ['gini', 'entropy']
max_depth = [9,10,11,12]
min_samples_split= [2,3,4,5]
best_score = 0
best_tree = None

for criterion in crit:
    for depth in max_depth:
        for min_samples in min_samples_split:
            tr = tree(criterion=criterion, max_depth=depth,
            min_samples_split=min_samples)
            tr.fit(X_train, y_train)
            cur_score = tr.score(X_test, y_test)
            if (cur_score > best_score):
                best_core = cur_score
                best_tree = tr

This is rather tedious and with more options to consider, the code can become unreadable. Luckily, scikit-learn provides a method of automatically doing this. All you do is provide a matrix of options and scikit learn will loop through all possible combinations to get the best fit for your data. It even provides the options that worked best.

from sklearn.model_selection import cross_val_score
from sklearn import tree
from sklearn.model_selection import GridSearchCV

X_train, X_test, y_train, y_test = train_test_split(x, y, test_size=0.2, random_state=2)

parameters = {'criterion':('gini', 'entropy'),
              'min_samples_split':[2,3,4,5],
              'max_depth':[9,10,11,12],
              'class_weight':('balanced', None),
              'presort':(False,True),
             }


tr = tree.DecisionTreeClassifier()
gsearch = GridSearchCV(tr, parameters)
gsearch.fit(X_train, y_train)
model = gsearch.best_estimator_

Ansible

2018-01-06T18:00:00+03:00

Ansible is an automation tool that easens work a lot. I just set it up once and from then on I can forget about configurations and settings for my projects. It's really great for doing repeatable tasks like setting up nginx for subdomains, redeploying django projects, updating the OS, etc.

For example, to set up a script that updates the server:

---

- hosts: my_server
   vars_files:
     - vars.yml
   remote_user: "{{ user }}"
   become: true
   tasks:
       - name: Running update and safe-upgrade
          apt:
              update_cache=yes
              upgrade=dist
          register: result
       - debug: var=result.stdout_lines

I store my variables in the vars.yml file. In the example above, the user variable is found here. The file looks like this:

---

user: thisuser

The my_server variable is set in /etc/ansible/hosts file and looks like this:

[my_server]
random_url.co.ke

Since updating the system requires admin priviledges, make sure the user has those permissions. This is why the script has remote_user and become in its body. Also ansible will need to run with these priviledges, so to do so:

ansible-playbook --ask-become-pass updates.yml

And just like that the system gets updated. This asks for the server password for the elevated user.

Let's say I want to update my nginx config files to enable another subdomain. Ansible should upload my config file to my server and also restart nginx so that these changes are accepted.

---
- hosts: my_server
   vars_files:
     - vars.yml

   tasks:
   # copy project nginx set up to server
     - name: set up nginx config on server
        remote_user: "{{ user }}"
        become: true
        template:
          src=config/nginx.j2
          dest=/etc/nginx/sites-enabled/{{ project_name }}
        notify:
          - restart nginx

   handlers:
     - name: restart nginx
        remote_user: "{{ user }}"
        become: true
        service: name=nginx state=restarted

Since ansible supports jinja templating, the nginx config file is written in this format. Also the variables project_name and user are found from the vars.yml file. What the above does is copy the nginx config file to /etc/nginx/sites-enabled/project_name restarts nginx.

Here is the template used. You can also use variables defined in the vars.yml in the template too, which is really helpful.

server {
#listen 443;
    listen 80;
    server_name {{ server_name }};
    charset utf-8;
    client_max_body_size 2M;

    location / {
        # django
        include proxy_params;
        proxy_pass http://unix:{{ sock_file }};
    }

    location /static {
        autoindex on;
        alias {{ static_root }};
    }

    location /media {
        autoindex on;
        alias {{ media_root }};
    }
}

Some commands can run with ansible-playbook file.yml but others need elevated permissions. To help me deal with this I usually have a Makefile in my ansible playbooks directory. For example to deal with the two configs above, I'd have:

ANSIBLE_SUDO = ansible-playbook --ask-become-pass

update:
    $(ANSIBLE_SUDO) updates.yml

nginx:
    #(ANSIBLE_SUDO) nginx_setup.yml

You can find moer about ansible here

Summary of 2017

2018-01-01T15:00:00+03:00

2017 was an interesting year in my life. I achieved a lot during this time despite starting off on a low note. It started off on a low note because following my graduation(2016), I was abit confused with my life. Things however became better as the year progressed.

I got my first job as a systems engineer at Koko Networks. I learnt a lot while I was there: circuit design, firmware development, managing and motivating teams and engineering operations in general. The team at Koko was really helpful and great. For that, I am grateful.

I also made my first production circuits this year. The first was a simple LED multiplexing array that was used to display various texts. The second was a simple auto-agriculture kit for Aura Engineering. These projects made me learn a lot about Kicad and how to deal with PCB manufacturers.

I was also an active contributor in the GrabCad community. I uploaded a total of 20 models in 2017. I have learnt a lot about using Solidworks and Inventor for design. I've experimented on using FreeCAD in doing some design work and I'm impressed by the many features it has. I want to do a lot more using FreeCAD because I'm a huge open source enthusiast.

In development, my first ever "serious" open source contribution was accepted and merged to the main source tree. It was to add support for rstcheck to vim-ale. Vim-ale is a vimscript linting engine. I also made a comic site system in django that currently runs on my server at comics.jnduli.co.ke. For big projects, I started on Mradi (an organisation system) which is still incomplete. Mradi's tech stack uses: VueJS, ansible, django in the process.

As far as infrastracture is concerned, I got to learn and use ansible. Writing playbooks is fun, and automating steps is an interesting challenge. Most of my projects currently have an ansible config that helps in set up and configuration of various tasks.

Reading this year has been fruitful. The highlight of my year in reading was completing the Foundation Trilogy by Isaac Asimov. It taught me a lot about politics, economics and society. I also have a huge catalog of unfinished books: Learn Vimscript the Hard Way, The Art of Unix Programming and Thinking in Systems. These are really great books that I should have completed.

My participation in Urban Perspective (a bookclub) and Nailug (Linux User Group) was also encouraging. I got to know so many interesting people, learn from them, critique some ideas and socialize.

Finally, I got the PAUSTI scholarship. This was a pretty trying moment for me because it meant I had to stop working and leave my colleagues at Koko. This was a tough decision to make and leaving Koko was a low point for me in 2017.

2017 was a great year. That was my summary of the year, and now I can see through 2018.

Zealot B570 Bluetooth Headset

2017-12-22T15:00:00+03:00

I got the zealot b570 a couple of months ago. I have been using it since then and it has not disappointed. In fact, I really like this headset. I'm also a bit clumsy, so it's been subjected to a rough time, and its still working well.

Pros

Bluetooth capable
SD card slot
OK battery life
I can listen to audio over mini usb and over 3.5 mm jack. I prefer using the mini usb option, so that's a plus.
Good audio quality
Really durable. I've never had a head set that lasted more than 3 months. But this was primarily due to the 3.5 mm jack getting cut / broken.
The control buttons are located at a good location.

Cons

The headset was really tight and painful to wear. To fix it, I stretched it for some days and it worked well.
The LED lights are really annoying. I wish there were an option to disable these.
I don't use the FM mode, so it kind of disappoints me when I'm switching modes and the noise from the FM mode comes up.

Project Management in Google Sheets

2017-12-16T08:00:00+03:00

Outline of Project

I wanted a google sheet file that could manage simple projects. This meant that it could track tasks, send notifications when tasks were late and differentiate done, undone and late tasks when the file was opened and editted. To do this, I decided to research into Google Sheets Scripts, which ended up being a powerful tool that I could use to achieve my means.

Sending notifications for Late Tasks

For this I first had to set a trigger. There are 2 methods of doing this that I currently know of.

Using the menu bar by going to: Edit > Current project's triggers > Add a new trigger
Programmatically by creating a function and running it once.

I went with the programmatic way because this would enable me to easier set it up on other sheets.

So assuming there is a function called mainFunction, I write the following trigger:

function createTimeDrivenTriggers() {
  ScriptApp.newTrigger('mainFunction')
      .timeBased()
      .everyDays(1)
      .create();
}

This will run the function 'mainFunction' once per day. To activate the trigger, click on Run > Run Function > createTimeDrivenTriggers. The trigger can now be seen in the triggers list menu. When running the trigger for the first time, a page will pop up requesting for some permissions to be allowed.

To send a mail using the script is also fairly easy. Since the project sheet might have may participants, I set up a function that sends emails to an array provided to it with a particular message.

function sendEmail(emails, message, subject){
  for (var i=0; i<emails.length; i++){
    MailApp.sendEmail(emails[i], subject, message);
  }
}

The emails are set in a sheet called 'lookup'. So the script will find this sheet and read the emails from it. The emails are stored in a column named 'Emails'.

function getEmailsToNotify(data) {
  var emailCol = getColumnByName(PropertiesService.getScriptProperties().getProperty('EMAILS_COLUMN'), data);
  var emails = [];
  if (emailCol < 0)
    return emails;
  for (var i=1; i<data.length; i++) {
    var email = data[i][emailCol];
    if (email.trim() !== ''){
      emails.push(email);
    }
  }
  return emails;
}

For the script, the preconditions I set for it to be able to run was that the first row should have the following names in the various cells:

Features
Status
DateDue

This means that each sheet has to follow this pattern. To enforce this I wrote the following script:

function getColumnByName(colName, data) {
  var col = data[0].indexOf(colName);
  return col;
}

function getNotificationsFromSheet(range) {
  var data = range.getValues();
  var dateDueCol = getColumnByName('DateDue', data);
  var featuresCol = getColumnByName('Features', data);
  var statusCol = getColumnByName('Status', data);
  if (dateDueCol < 0 || featuresCol < 0 || statusCol < 0 ){
    return ["error : Sheet does not follow expected format"];
  } else {
    messages = getLateDateDues(dateDueCol, statusCol, featuresCol, range);
    return messages;
  }
  // checkLateDateDues(dateDueCol, statusCol, featuresCol, data);
  // return ['this is cool', 'this is another one', 'I love this'];
}

The function getColumnByName checks the first row for any columnName provided to it. If a columnName is not found it returns -1. So in the function getNotificationsFromSheet, we check for the 3 cells, and if there is any that is missing and error is returned.

The getLateDateDues function just loops through all the dates comparing them to the current date. If any date is late and its status is not 'done', it adds this to the messages for alerting and also colors that particular row red.

function getLateDateDues(dateDueCol, statusCol, featuresCol, range) {
  // loop through the date due column only
  var data = range.getValues();
  var dateNow = new Date();
  var messages = [];
  for (var i =0; i<data.length; i++) {
    var dateDue = new Date(data[i][dateDueCol]);
    var rowRange = range.offset(i, 0, 1);
    if (dateNow > dateDue && data[i][statusCol] !== 'done'){
      messages.push(data[i][featuresCol] + ' was due on ' + dateDue);
      rowRange.setBackground('#ffcdd2');
    }
  }
  return messages;
}

You can find a copy of the complete gist here:late

Changing Colors while Editting

While editing the file, you also need to get some visual feedback on deadlines and errors. To do this, google sheets provides simple triggers that help out. For example, you can try this out:

function onEdit(e) {
  Logger.log('The sheet has been editted');
}

If you save this, and edit any cell on the sheet, you will see that message on the logger. To access the logger : View > Logs

The first thing to check for is if the editted cell belongs to the DateDue or status column. To do this, you need to get the activeSheet and the DataRange.

function onEdit(e) {
    var activeSheet = e.source.getActiveSheet();
    var sheetName = activeSheet.getName();
    var range = activeSheet.getDataRange();
}

Then we can get the column editted and row editted. Note that e.range.getColumn() and e.range.getRow(), start their indexes at 1, so to work properly with arrays we need to subtract 1 from the values we get.

function editTasksSheet(range, e){
  var data = range.getValues();
  var columnEditted = e.range.getColumn()-1;
  var rowEditted = e.range.getRow()-1;
  var valueInput = data[rowEditted][columnEditted];
  var dateDueCol = getColumnByName('DateDue', data);
  var statusCol = getColumnByName('Status', data);
  if (dateDueCol < 0 || statusCol < 0){
    return;
  } else if (columnEditted === dateDueCol) {
    var rowRange = range.offset(rowEditted, 0, 1);
    // var dateInput = data[rowEditted][columnEditted];
    formatBasedOnDate(rowRange, valueInput);
  } else if (columnEditted === statusCol) {
    if (valueInput === 'done') {
      var rowRange = range.offset(rowEditted, 0, 1);
      rowRange.setBackground('#c8e6c9');
    } else {
      formatBasedOnDate(rowRange, data[rowEditted][dateDueCol]);
    }
  }
}

From the above code snippet, we first look for the columns named 'DateDue' and 'Status'. If they are not existent, the script stops working because the sheet is not properly formatted. Afterwards we just format the sheet appropriately based off these rules:

If the date input is older than the current date, color the row red since it is a late task
If date input is later than current task, reset the background color to white.
If status input is done, color the row green
If status is something other than done, do the date formatting.

The completed file can be found in the following gist here:edit

Prepare Kicad PCB for Elecrow Production

2017-12-07T08:00:00+03:00

Before designing the PCB, set the following design rules:

Minimum PCB Track : greater than 0.2mm
Minimum Track /Vias space: greater than 0.2mm
Minimum pads space: greater than 0.2mm

The defaults provided by Kicad are OK, so no need to change them unless you have some special requirements.

Once the pcb has been completed, go go:

File>>Plot

Make sure the following options are checked in Layers section:

F.Cu
B.Cu
B.SilkS
F.SilkS
B.Mask
F.Mask
Edge.Cuts

Make sure the Plot format is set to Gerber. It is also advisable to set an OutputDirectory for your gerber files. This prevents mixing up the gerber with normal project files.

Make sure the following are checked in Options:

Plot footprint values
Plot footprint references
Exclude PCB edge layer from other layers

For Gerber options, make sure this is checked:

Use Protel filename extensions

Click on Plot to generate the gerber files.

To generate the drill files, click on "Generate Drill File". I set the Drill Units to Millimeters, Zeros Format to Decimal Format and Dril Map File Format to Gerber. Also Drill Origin was set to Absolute. Then click Drill File.

Kicad outputs files with a different naming convention as that expected by elecrow. So you'll have to rename these files. For examples:

pump-B.Cu.gbl becomes pump.gbl
pump-B.Mask.gbs becomes pump.gbs
pump-B.SilkS.gbo becomes pump.gbo
pump-Edge.Cuts.gm1 becomes pump.gml
pump-F.Cu.gtl becomes pump.gtl
pump-F.Mask.gts becomes pump.gts
pump-F.SilkS.gto becomes pump.gto
pump.drl becomes pump.txt

After renaming the files you can then compress all of them into a zip file and make the order.

ArchLinux Signature Error

2017-11-30T20:00:00+03:00

When updating archlinux using:

sudo pacman -Syu

I sometimes get an error message that looks something like this:

error: zstd: signature from "Andrzej Giniewicz (giniu) <randomemail@gmail.com>" is marginal trust
:: File /var/cache/pacman/pkg/zstd-1.3.2-1-x86_64.pkg.tar.xz is corrupted (invalid or corrupted package (PGP signature)).
Do you want to delete it? [Y/n] Y
error: failed to commit transaction (invalid or corrupted package (PGP signature))
Errors occurred, no packages were upgraded.

Pacman uses a collection of PGP keys which check signed packages and databases. If a key is corrupted or invalid, a package cannot be installed or updated. In the example presented above, the package zstd prevented my whole system from getting upgraded.

To fix this, you try to update the keys using the following:

sudo pacman -Sy archlinux-keyring
sudo pacman-key --populate archlinux
sudo pacman-key --refresh-keys

The source for this fix is from here:source

Using Pass - A Password Manager/Generator

2017-11-16T14:00:00+03:00

Pass is a password manager that stores passwords locally on the host machine. Since the passwords are stored locally there is minimal risk on your passwords being exposed after some data breach occurring. Also, you can use git to version control the passwords, and set them up in a repo on any machine.

To install:

sudo pacman -S pass

Then to initialize the password store:

pass init email

To insert a password into the store:

pass insert /email/email@dns.com

If you get the following error:

gpg: email@dns.com: skipped: No public key
gpg: [stdin]: encryption failed: No public key
Password encryption aborted.

Then you have to first generate the gpg keys:

gpg --full-gen-key

This will take you through the process step by step. All someone has to do is follow the prompts.

To generate a password, do:

pass generate name/of/service 10

which will generate a password 10 characters long

pass -n generate name/of/service 10

will generate password without special symbols i.e. only alphanumeric characters.

To edit or overwrite password

password edit name/of/service or pass inserte

Other commands include:

pass name/of/service # displays the password
pass -c name/of/service # copies the password to xclip

pass rm name/of/sercie # removes password
pass rm -r name # removes full directory

pass git init # sets up git in pass folder

To find out more about pass, you can check here

Keeping a Diary in Vim

2017-11-05T09:00:00+03:00

A diary is a good way to write down one's thoughts and memories of the day.

Being a vim fan, I had to find a way to easily help me write down my diary. This is where vim wiki comes in. It is basically a means of writing down wikis in vim, and has an awesome diary mode.

I use vundle as my vim plugin manager. So to install vimwiki, I added to the following to my vimrc file:

Plugin 'vimwiki/vimwiki'

And then ran the following from withing vim (while the vimrc file is the current buffer):

:source %
:PluginInstall

To enter diary mode, I just type <leader>wi in any vim session and it will direct me into the diary entries.

To add a diary on the day's happenings, I just type <leader>w<leader>w and it will open a blank buffer. When this is saved it will be a new day's entry.

To update the diary with the new entry, <leader>wi will take you to the page containing the various entries. Then type <leader>w<leader>i and this will update the list with the new entry.

Since I use vim a lot, I find this really useful when collecting my thoughts during a day.

So in summary these are the commands helpful for vimwiki diary mode:

<leader>w<leader>w " create new diary note for the day
<leader>wi " show all diary entries
<leader>w<leader>i " update entries

Scrolling Text With 5*4 LED Array

2017-10-29T10:20:00+03:00

I usually see lights in Nairobi scrolling text from right to left. It is an interesting thing to see, and if done right is usually beautiful. I wanted to make one, and so this is how I went about it.

Multiplexing LEDs

To light and LED one needs 5V supplied to cathode and GND supplied to anode. This means if we have 2 leds, both receiving 5V and only one receiving GND, only one LED will light. The same occurs when both receive GND and only one receives 5V. This is the basic building block of an LED matrix.

So how does one keep this so. Well, one option is using a microcontroller. The microcontroller's pins can be set to output a HIGH (5V) or a LOW(GND/0V). So having a microcontroller will help us out in choosing which LED to use.

The following is the schematic I then came up with:

To have the scrolling effect, I then wrote code that turns on and off LEDs at a particular pattern. Making this faster will result in less flickering of light. I used an atmega 8 for this and added the respective current limiting resistors. The circuit and code for this can be found here:githublink.

Here is the image of the final fabricated circuit:

You can find the video of the working circuit here. The text being shown is "HAPPY EASTER" in case it is the video is not clear enough.

I3 Window Manager

2017-10-21T08:00:00+03:00

I've been using xfce4 as my primary window manager for some time now. Its pretty light weight and works really well on my laptop. I sometimes use an Acer D270 with 2GB Ram and 1.6Ghz processor, which is weak.

However, I decided I wanted to try out something new. I'd never actually used another window manager in linux apart from the defaults provided by various distros and xfce4 ( which I always install). So I did some digging around and found i3.

i3 is a tiling window manager, and my current default window manager too. I don't know if its the greatest one or even a good one. It was just the first other window manager I tried after xfce4 and I liked it a lot. In fact, I liked it so much that I haven't tried tinkering with another window manager yet.

To install i3 in archlinux:

sudo pacman -S i3wm i3lock i3status dmenu

The default i3 config file works but I had to make some of my changes.

First off, I really like the xfce4-terminal, so I wanted to still use it as my default terminal. This terminal also has a drop-down feature which I use a lot, so I also had to figure out a way to do this. So this was my change to support this:

bindsym $mod+Return exec xfce4-terminal
# xfce4 dropdown terminal
bindsym F12 exec xfce4-terminal --drop-down
for_window [class="xfce4-terminal"] floating enable

Since I'm a dedicated vim user, I also had to find a way to customize i3 to work similarly to vim.

# change focus
bindsym $mod+h focus left
bindsym $mod+j focus down
bindsym $mod+k focus up
bindsym $mod+l focus right

# move focused window
bindsym $mod+Shift+h move left
bindsym $mod+Shift+j move down
bindsym $mod+Shift+k move up
bindsym $mod+Shift+l move right

There are also some custom applications I use, and that I wanted to start up with i3. An example is redshift. To do that I just added this to the config file.

# redshift
exec --no-startup-id redshift-gtk

To take a screenshot, I decided to use scrot. Here is my addition to i3 for that.

# scrot screen shot
bindsym $mod+p exec "scrot ~/Downloads/Screenshot%Y-%m-%d%H:%M:%S.png"

To change the desktop wallpater, I decided to use feh.

# background image
exec feh --bg-scale ~/Pictures/i3_solarized.png

To use i3lock, add the following to the config: .. code-block:: bash

#setting up i3lock exec --no-startup-id xautolock -time 1 -locker 'i3lock -n'

I'll be making a lot of more changes to my i3 config file. To checkout what I've implemented or changed you can view the latest config file here

Disabling Blanking in Raspberry Pi

2017-10-17T15:00:00+03:00

Blanking is whereby a screen just goes blank (i.e. black screen). This usually happens after some time of no usage on the raspberry pi which is about 15 minutes. This can sometimes become frustrating especially if it is a screen set up for monitoring something.

So to disable it temporarily:

xset s off # disables screen saver
xset -dpms #disable power management settings
xset -s noblank

To disable it permanently, open the file /etc/lightdm/lightdm.conf:

sudo vi /etc/lightdm/lightdm.conf

Then add the following line:

xserver-command=X -s 0 -dpms

GRUB RESCUE FiX

2017-09-21T15:00:00+03:00

So I found myself in windows some time this week, and it started updating. The process had one major consequence, grub could not discover my linux partition, and so brought me to a prompt containing 'grub_rescue'.

To fix this, I first did:

ls

This will show a list of the partitions of my hard drive, something like:

(hd0) (hd0,gpt2) (hd0,gpt3) (hd0,gpt4)

I needed to get the partition with my linux on it. So to do this I did:

ls (hd0,gpt2)/
ls (hd0,gpt3)/

While doing this, I was looking for a partition that has folders similar to those in root. So when I got a partition that has bin, boot, home, lib, etc in it, I knew I had got the correct one.

Assuming its (hd0,gpt3), I then did:

set root=(hd0,gpt3)
set prefix=(hd0,gpt3)/boot/grub
insmod normal
normal

With this, I was able to load the normal grub menu and boot into archlinux.

Tox this permanently:

mount /dev/sda1 /boot/efi
sudo grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=grub
sudo grub-mkconfig -o /boot/grub/grub.cfg

This is just from my steps on installation of archlinux on my asus laptop found here

Asus Zenbook Keyboard & Screen Brightness Fix On i3

2017-09-01T14:00:00+03:00

Keyboard backlight fix

Source for this information can be found here

The link has a file called kb-light.py. I set up this file in /usr/local/bin.

I then opened my i3config file and added the following:

bindsym XF86KbdBrightnessUp exec python /usr/local/bin/kb-light.py +
bindsym XF86KbdBrightnessDown exec python /usr/local/bin/kb-light.py -

And now when I press the keyboard brightness up and down key, the keyboard lights change as expected.

Screen brightness fix

For this I wanted to use xbacklight. The problem I got was that when I tried

xbacklight -inc 10

I got the error "No outputs have backlight propery".

To fix this, I installed the intel driver

sudo pacman -S xf86-video-intel

And after rebooting it worked well. So to set up the keyboard keys for screen brightness, I added this to my i3 config file:

bindsym XF86MonBrightnessUp exec xbacklight -inc 10 # increase screen brightness
bindsym XF86MonBrightnessDown exec xbacklight -dec 10 # decrease screen brightness

Media Keys Usage Fix

For this, I first had to install pulseaudio.

sudo pacman -S pulseaudio

Then I added this to my i3 config files:

bindsym XF86AudioRaiseVolume exec --no-startup-id pactl set-sink-volume 0 +5% #increase sound volume
bindsym XF86AudioLowerVolume exec --no-startup-id pactl set-sink-volume 0 -5% #decrease sound volume
bindsym XF86AudioMute exec --no-startup-id pactl set-sink-mute 0 toggle # mute sound

Now audio can be increased / decreased using my media keys.

My i3 config file can be found here:i3_config_file

My First Vim Function

2017-08-09T08:20:00+03:00

I finally got down to writing my first vim function. It was a gruelling journey but I finally got through it. So I'll write down the problem I had and the fix I finally came up with as my solution.

Problem Statement

I usually keep a to do list of things I should do during the day and week. Over time, tasks get lost as I remove some lines and I can't keep a track of my progress in terms of planning and implementation. So basically I wanted a means for later on in the year or in my life, I could have a file that showed how much I planned and how much I had achieved.

So I created the file. It had the following syntax. I use vimwiki, so to generate the table I just type:

:VimwikiTable 5 3

The file looks like this:

| Week | Total | Achieved | %    | Comments                        |
|------|-------|----------|------|---------------------------------|
| 1    | 13    | 1        | 7.69 | setting up lap  took up time    |
| 2    | 20    | 9        |      |                                 |

This table has a percentage column that I wanted to be automatically calculated. This would provide a means of checking whether I'm becoming better at goal setting and goal achievement over time.

Implementation

To get the percentage, I had to basically get the row value that corresponded to total, and the row value that corresponded to achieved.

To get the once corresponding to totaL, I did this in normal mode:: ^3wviw

What this does is this:

^ goes to beginning of line 3w goes 3 words ahead after going to beginning of line viw selects the current word, which happens to be the total

With this I had the number in total column selected, so I had to find out how to store the number for later use. Thats when registers came into play. So adding:

"py

Will yank the selected word and store it in the named register p. With this I can get the variable from the register. So to get the number 13 in the first row, this was the function:

function! TodoPercentage()
    normal ^3viw"py
    let total = str2float(getreg("p"))
endfunction

The getreg() function just gets the values stored in a register, and the str2float function converts a string to a float variable.

To get the achieved value, I also did the same.

function! TodoPercentage()
    normal ^3wviw"py
    let total = str2float(getreg("p"))
    normal ^5wviw"py
    let achieved = str2float(getreg("p"))
endfunction

Since I had these two variables, I could now do the arithmetic.

let percentage = (achieved/total) * 100

And since I just wanted to display the number in maximum of 2 decimal places, I used printf command, which sets the variable to 2 decimal places and also saves it in the per variable as a string.

let per = printf("%.2f", percentage)

To display the number in the appropriate section, I decided to use the execute command. This would execute the string, converting special characters to vim commands e.g. <esc> to actually pressing escape. The "normal!" executes the commands in normal mode, the ! eliminates the custom mappings made.

execute "normal! ^6wa ".per."\<esc>"

After the "normal!" the following happens:: ^6w : moves to the start of the line, then 6 words after a : appends characters, enter insert mode .per. : concatenates the percentage string. Thus it will be typed character by character "<esc>" : initiates the esc sequence leaving insert mode. The is used to signify that the key ESC is what is meant and not the actual characters.

So at the end of it all this was my function:

function! TodoPercentage()
    normal ^3wviw"py
    let total = str2float(getreg("p"))
    normal ^5wviw"py
    let achieved = str2float(getreg("p"))
    let percentage = (achieved / total) * 100
    let per = printf("%.2f", percentage)
    execute "normal! ^6wa ".per."\<esc>"
endfunction

Now to run this function I decided to map it to <leader>cp meaning calculate percentage.

autocmd FileType vimwiki nnoremap <leader>cp :call TodoPercentage() <Cr>

You can find this function implemented in my .vimrc file here

Setting Up Raspberry Pi

2017-07-16T08:20:00+03:00

The raspberry pi is a mini computer with GPIO pins that can be used to interface with various sensors and actuators. More information of the raspberry pi can be found here

The SD card should be of a size greater than 4GB, especially if you want to install Raspbian Jesse.

Connect the card to your computer and get its id. To do this:

sudo fdisk -l

If you do the above command before inserting the card and after inserting the card, you'll find a new addition which is the card.

Format the card to fat partition. To do this:

sudo umount /dev/sdcardfound
sudo mkfs.fat /dev/sdcardfound

After this the sd card is ready for use.

Download the OS from here:raspbian, and unzip it. I chose Raspbian Jesse with Desktop for my OS.

unzip downloadedraspbian.zip

Then burn the OS into the sd card using dd:

sudo dd if=unzippedraspbian.img of=/dev/sdcardfound bs=4M status= progress

Once this step is completed the OS is ready for use.

With Screen, Keyboard and Mouse

Attach the sd card, screen, keyboard and mouse to the raspberry pi. For power, a supply rated 5V 2.5A is recommended. In my case, I used one rate 5V and 1A and it worked well. When powered, there are two LED lights, a red one and a green one. The red should be on indicating the pi is receiving power. The green light should be blinking. If it is not this indicated either a problem with the sd card or the power supply.

Once the pi is started, wait until the desktop environment is loaded. Then you can then enable vnc and ssh.

To do this, go to the terminal and type:

sudo raspi-config

Go to interface and enable both vnc and ssh.

You can also go to MENU > PREFERENCES > Interfaces. From this you can also enable ssh and vnc.

To set up a virtual keyboard, first connect to a wifi/LAN network and do:

sudo apt update
sudo apt install matchbox-keyboard
sudo reboot

The keyboard should be in accessibility menu. If it isn't go to MENU > Preferences > Main Menu Editor > Accesorites. Disable and enable the Keyboard option.

To ssh into the pi, first get the ip address. To do this, hover on the wifi icon and the ip will be displayed. You can also get the ip address from the terminal by running:

ip address show

From another computer, run:

ssh pi@piipaddress

The password is : raspberry

The pi already comes with a vnc server known as realvnc. So just install the viewer on your laptop and use it. For archlinux:

yaourt -S realvnc-vnc-viewer

Then confirm that vnc is enabled on the pi. To use it:

vncviewer piipaddress

Realvnc can also be setup to run via the cloud. Just visit their website here:realvnc to find out how.

ASUS ZenBook UX330UA ArchLinux Setup

2017-06-28T15:00:00+03:00

I recently got the ASUS ZenBook UX330UA-AH54, which is a really awesome machine. I've been using it for one month now, and its beautiful.

So first thing I did with the laptop was install archlinux (dual boot with windows). These are the steps I took to set up archlinux. Most of these steps can be found in one form or another on the archlinux wiki

Hard Drive Preparation

I partitioned the hard drive using windows tools.

I then downloaded archlinux iso file in another machine that had archlinux, and attached a flash drive. To setup the iso file, I did this from the terminal:

dd if=archlinux.iso of=/dev/sdb bs=4M status=progress

I then connected this flash drive to the Asus laptop and did a reboot. When the Asus logo shows, I pressed <Esc> so that I could choose which bootable media to use. I selected the flash drive and unfortunately got the error:

Invalid signature detected. Check secure boot policy in setup.

I then went to set up and disabled secure boot policy.

I successfully managed to enter the archlinux environment. I verified uefi mode by using:

ls /sys/firmware/efi/efivars

This gave me some content, thus it was fine.

Internet Connection

I connected to the internet via wifi. To do this:

iw dev #to get name of wireless interface. I got wlp2s0
ip link set wlp2s0 up #I had to activate the kernel interface
ip link show wlp2s0 #To check if activated. There was an UP between the <> info
iw dev wlp2s0 scan | less #list wifi Access Points in area
wpa_supplicant -i wlp2s0 -c <(wpa_passphrase "SSID" "keytoaccesswifi") -B #connect to wifi
iw dev wlp2s0 link #to check if associated
dhcpd wlp2s0 #to get up address from wifi

System Clock

To update the system clock, I did:

timedatectl set-ntp true
timedatectl status # checks the time

Initial Setup

To install arch, I had to view the partitions and the sizes so that I could choose the correct one.

fdisk -l

After getting the correct one:

mkfs.ext4 /dev/sda4 #format partition as ext4
mount /dev/sda4 /mnt #mount partition to start working with it

I then set up some basic packages for arch:

pacstrap /mnt base base-devel

I generated the fstab:

genfstab -U /mnt >> /mnt/etc/fstab

Now change into the new file system root:

arch-chroot /mnt

I set the time zone for my system:

ln -sf /usr/share/zoneinfo/Africa/Nairobi /etc/localtime
hwclock --systohc

For the time zone once '/user/share/zoneinfo' has been type, clicking tab will autocomplete.

I set the locale by uncommenting the 'en_US.UTF-8 UTF-8' and the 'en_GB.UTF-8 UTF-8' lines in the /etc/locale.gen file and then ran:

locale-gen

To set the default locale, I create a file /etc/locale.conf and added LANG=en_US.UTF-8 to it.

echo LANG=en_US.UTF-8 >> /etc/locale.conf

The Hostnames file /etc/hosts was already existing so I did not edit it. I then set the root password by running:

passwd

Bootloader installation

I first installed grub and efibootmgr.

pacman -S grub efibootmgr

I needed parted so that I could know my ESP partition ( EFI system partition). The actual partition will have the name shown. So to do this:

sudo pacman -S parted
parted /dev/sda print # to know which was my ESP partition

After getting the ESP partition, I mount /boot/efi onto it and install grub.

mount /dev/sda1 /boot/efi
grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=grub
pacman -S intel-ucode #enabling microcode updates
grub-mkconfig -o /boot/grub/grub.cfg

I then reboot the machine. I get an error at the login prompt but it can be ignored.

User Creation

To create a user:

useradd -m -G wheel -s /bin/bash username
passwd username

I then set up sudo on the laptop:

pacman -S sudo
EDITOR=vi visudo #and uncommnet the line %WHEEL  ALL= (ALL) ALL

The line containing "%WHEEL ALL= (ALL) ALL" is uncommented.

Window Managers and Other Applications

I first set up xdg-dirs, and the run xdg-user-dirs-update. I then open the file .config/user-dirs.dirs and set the values to how I want my home directory to be organised.

I then copy the /etc/xdg/user-dirs.code file to the .config folder and set enabled to false.

I then create all my directories as pwer user-dirs.dirs manually using mkdir.

sudo pacman -S xdg-user-dirs
run xdg-user-dirs-update
vi .config/user-dirs.dirs
cp /etc/xdg/user-dirs.conf ~/.config/
vi .config/user-dirs.conf #change enabled=False

I then install vim and tmux and configure it as per the link .

sudo pacman -S gvim tmux

I also set up powerline fonts as found here to help in vim_airline setup.

I then install zsh:

sudo pacman -S zsh zsh-completions

And then setup oh-my-zsh as per the instructions here: zshlink

I then install xorg group, sfce4 and xfce4-goodies for one of my desktop environments.

I also install i3 as my main window manager.

Other Installs

sudo pacman -S python python2 python-pip python2-pip
sudo pacman -S kicad kicad-library kicad-library-3d
sudo pacman -S openssh
sudo pacman -S mtp gvfs-mtp

I also use ledger to manage my finances so I have to install it. Ledger is only found in AUR so I first installed yaourt from instructions found here :yaourt

After that I install ledger with:

yaourt -S ledger

The following installs are requirement for my i3 config to work properly:

sudo pacman -S redshift feh scrot dmenu
sudo pacman -S python-gobject python-xdg librsvg #required for redshift-gtk

For xfce4, I added the following shortcuts:

exo-open --launch TerminalEmulator  Super+Return
xfce4-terminal --drop-down F12

To switch capslock and escape, I added this to my .xinitrc:

setxkbmap -option caps:swapescape

The audio by default was muted so to fix this I did the following:

sudo pacman -S alsa-utils
alsamixer
#went to master and unmuted by pressing m

And with that I was comfortable enough with my archlinux setup.

Vim Airline Setup

2017-06-12T08:20:00+03:00

I was watching this video on how to use vim+tmux effectively (Video Link), and this guy had some really awesome looking vim. I decided to try it out and see if I could end up with something just that great.

I did my research and found two alternatives, powerline and vim-airline. Powerline is pretty heavy and vim-airline states itself to be lightweight, so I went ahead and installed vim-airline.

"Added so that vim airline can get git info
Plugin 'tpope/vim-fugitive'
Plugin 'vim-airline/vim-airline'

After this I just run :PluginInstall.

However, after installation, the statusline did not look that great. I needed to do something more. I foundout that vim-airline needs some special fonts so that it can display some things properly, like the > and <. To do this, I had to install powerline fonts. I followed the instruction found here: powerine fonts and ended up installing the powerline fonts.

I then switched to a powerline font in Xfce4-terminal. I use DejaVu Sans Mono for Powerline Book.

But still this was not enough. I then had to go to my .vimrc and add some few more sections.

"To allow colors in the status line
set t_Co=256
"To allow status bar to be seen even before splitting
set laststatus=2
"To allow powerline symbols to be seen
let g:airline_powerline_fonts=1

With all this I had almost reached my goal. However, I have a 10.1" screen laptop, and the status bar seemed to be hiding some details because of the little screen landscape. So I decided to change some sections. After some digging I added this to my .vimrc

"Reduces the space occupied by section z
let g:airline_section_z = "%3p%% %l:%c"

And now I have a pretty decent status bar.

Git Branches

2017-06-04T15:00:00+03:00

Lets say we have have a project A that works really well and serves our purpose. Then there is this new killer feature that we want to add to the project. Adding it directly to project A is not a really good thing as it might break whats working.

So what happens. We create a branch on which we start implementing this new feature. The branch is a copy of the code, that we can work with safely. The branch can be experimental features or just things we want to test to see if they are stable enough.

Branches are an important feature in git ahd here are the basic commands I use:

git branch #list all branches
git branch <name> #creates branch called name
git branch -m <name> #change branch name to name
git branch -d <name> #deletes the branch. If changes havent been
                    #merged an error message is shown.
git branch -D <name> #force delete branch even with unmerged conflicts
git checkout <existing-branch> #switch to this branch
git checkout -b <name> #create branch name and switch to it

Sometimes you create a branch but it cannot push to the server. To fix this, you do:

git push --set-upstream origin <branch_name>

And when you clone from a server, the git branch command will only show the master branch. To view all the branches you do:

git branch -a

These are the branch commands I interact with most of the time. I'll add more over time as I get more familiar with branching.

Adb Usage at Work

2017-04-09T08:20:00+03:00

To use adb in archlinux, first install android-tools.

sudo pacman -S android-tools

I then add the adb resources to the android rules files. To do this, first do lsusb, you'll get something like this:

lsusb
Bus 001 Device 004: ID 12d1:256b Huawei Technologies Co., Ltd.

To add this to the rules, open up /etc/udev/rules.d/51-android.rules And add this:

SUBSYSTEM=="usb", ATTR{idVendor}=="12d1", MODE="0660", GROUP="plugdev"

To list all the packages installed:

adb shell pm list packages -f

To get the main activity of a package

adb shell pm dump com.randompackage.android.application | grep -A 1 MAIN

To launch activities

#launches the launcher activity
adb shell monkey -p app.package.name -c android.intent.category.LAUNCHER 1

or:

adb shell am start -n com.package.name/com.package.name.ActivityName

To start a specific action

am start -a com.example.ACTION_NAME -n com.package.name/com.package.name.ActivityName

To install apk

#if not previously installed
adb install filename.apk
#if it had been installed eg. when upgrading
adb install -r filename.apk

Why To Use Github

2017-03-19T10:20:00+03:00

Github is a really awesome tool. It's where most open source project live. In my case, I use github as a means of control and tracking. I've been doing so many projects in my free time, but however I do not have a record or something that can show what I've been doing. For example, I made a game a couple of years ago, but somehow lost the code. So I can't actually state that I've made a game.

Github too works really well with Kicad. Kicad is an electronic schematic capture and pcb design tool. It works well because kicad stores all of its files in text format. Thus github is a good way to manage my circuits.

Github, I believe will also help me keep a track of where I'm coming from and where I'm going. I've discovered recently that using github make me use it even more.

Postgresql Weird Error - Failed To Start

2017-03-12T19:44:00+03:00

After updating my Archlinux, I tried starting up postgreSQL with:

sudo systmctl start postgresql.service

The service was unable to start. I then checked on what systemctl was saying the error was.

(env) [rookie@ArchRookie eventmanagement]$ sudo systemctl status postgresql
[sudo] password for rookie:
● postgresql.service - PostgreSQL database server
Loaded: loaded (/usr/lib/systemd/system/postgresql.service; disabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Sun 2017-02-19 17:43:52 EAT; 3min 24s ago
Process: 4210 ExecStartPre=/usr/bin/postgresql-check-db-dir ${PGROOT}/data (code=exited, status=1/FAILURE)
Feb 19 17:43:52 ArchRookie systemd[1]: Starting PostgreSQL database server...
Feb 19 17:43:52 ArchRookie systemd[1]: postgresql.service: Control process exited, code=exited status=1
Feb 19 17:43:52 ArchRookie systemd[1]: Failed to start PostgreSQL database server.
Feb 19 17:43:52 ArchRookie systemd[1]: postgresql.service: Unit entered failed state.
Feb 19 17:43:52 ArchRookie systemd[1]: postgresql.service: Failed with result 'exit-code'.

I then decided to check via journalctl.

(env) [rookie@ArchRookie eventmanagement]$ journalctl -xe
-- The result is failed.
Feb 19 17:43:52 ArchRookie polkitd[321]: Unregistered
Authentication Agent for unix-process:4201:1104590 (system
Feb 19 17:43:52 ArchRookie systemd[1]: postgresql.service: Unit
entered failed state.
Feb 19 17:43:52 ArchRookie systemd[1]: postgresql.service: Failed
with result 'exit-code'.

This too was not descriptive enough. I then went to the archlinux wiki docs and found a solution. Apparently something had messed up my '/var/lib/postgres/data' and so this was my fix:

su root
su postgres
initdb --locale $LANG -E UTF8 -D '/var/lib/postgres/data'

This is the Source Link .. Source: https://bbs.archlinux.org/viewtopic.php?id=194040

Solarized Theme

2017-02-26T08:20:00+03:00

When using a tool or anything for a really long time, one has to really like how it looks like. I use the terminal, vim and tmux a lot. So I had to look for a theme that would be easy on my eyes. A quick internet search revealed solarized, which I then went on to set up. I have never looked back since then.

Xfce4 terminal setup

Go to Preferences>Colors

Then change the terminal colors to "solarized-dark"

Vim setup

I use vundle as my plugin manager in vim. So setting up solarized was as easy as adding the following line:

Plugin 'altercation/vim-colors-solarized'

And then running:

:PluginInstall

After these I added the following lines to my .vimrc:

syntax enable
set background=dark
colorscheme solarized

And vim looked great. More configuration settings can be found here: solarized-github-page

Tmux setup

I use the tmux plugin manager (tpm-link). So to setup solarized on tmux, it's as simple as:

set -g @plugin 'seebi/tmux-colors-solarized'

Then I run prefix + I , whicn in my case it Ctrl+b and then I press I.

This will download and install the tmux solarized theme.

After this I add the following line to my tmux config:

set -g @colors-solarized 'dark'

For more information of tmux solarized config, check here: tmux-solarized

Know Thy Editor Well

2017-02-19T08:20:00+03:00

Someone once told me that I should really know my editor, and know it well. I did not take this advice into consideration but lately, as I get better and better at using vim, I think I'm beginning to understand the fella well.

If you know your editor well enough, it becomes the single point of reference for every task you will try to do. And the more you know your editor, the better it works for you. Also, your performance improves significantly as you become more comfortable with your editor.

If you become comfortable with the editor, you might end up using it for the most importabt aspects of your life. It can end up integrating into a lot of really cool stuff you use.

Here is a list of some of the things I use vim for: * Managing my finances, works together with ledger * Writing a diary/journal of my life * Note taking using vimwiki * Todo lists using vimwiki and/or vim orgmode * Coding -- obviously * Blogging * Server setup and management * Documentation * Make presentations

As can be seen, knowing vim well will greatly optimize my time and also effectively manage my life well. It should not only be vim, but know that whatever editor you choose to use, know it well, know its community and learn every day how to optimize your work.

Changing Menu Icon MinCss

2017-02-12T08:55:00+03:00

I wanted to make this pelican theme that is really light weight. I did some research and came upon mincss. Now this is a really small CSS framework that supports a wide range of browsers. For more info, check out here

I encountered a small problem however. I wanted to change my menu background colour. By default, micsss menu has a black backgroun and a menu icon that is white in colour.

So I thought they used an icon or image, only to find out mincss has this really neat treat of using borders. The following line shows its implementation:

.nav div:before {
    background: #000;
    border-bottom: 10px double;
    border-top: 3px solid;
    content: '';
    float: right;
    height: 4px;
    position: relative;
    right: 3px;
    top: 14px;
    width: 20px;
}

This code creates three lines with the border command thus the menu icon.

To make it appear well in my theme, ie. in a white background with black lines, I just changed the code to this.

.nav div:before {
    background: #fff;
    <!--everything else remains the same-->
}

I then added a bit of snippets on my own to make the background white.

.nav{
    background: #ffffff;
}

And thus I had a menu I pretty much liked.

Tmux Usage

2017-02-05T08:20:00+03:00

I use xfce4-terminal which is really awesome. It light on my computer and yet heavy on features. It has tabs, a drop down terminal option, and many more.

However, I soon came to realize I needed something more. Something that I could use with together with the features of xfce4-terminal. That's when I got to find tmux.

Tmux put simple is a terminal multiplexer, meaning it gives you multiple terminals within the same window i.e kinda like tabs but more powerful. You can have multiple windows on the same screen(it works like vsplit and hsplit of vim) and multiple windows in the same session.

To make you understand, I just have to show you how to use it:

sudo pacman -S tmux

To start a tmux session:

tmux

After than you can create mutliple windows on screen or in buffers

C-b c " creates a new buffer
C-b , ' renames a new buffer
C-b number "moves to the numbered buffer
C-b " 'Creates a vsplit
C-b % "creates a hsplit

To move through sections I wanted to use vim directional buttons h,j,k,l so I had to configure this in tmux config file. So I created a .tmux.conf in my home folder and added the following:

bind -r k select-pane -U 
bind -r j select-pane -D 
bind -r h select-pane -L 
bind -r l select-pane -R

So if I have multiplw window splits on screen, I just do:

C-b h/j/k/l " move to window

I also set the resizing of split to use

bind -r C-k resize-pane -U
bind -r C-j resize-pane -D
bind -r C-h resize-pane -L
bind -r C-l resize-pane -R

So to resize a window split I just do:

C-b C-h/j/k/l

If you have multiple windows and have exited from one, the numbering is usually flawed ie. it can be 1,2,4,5. To renumber the windows, I do

C-n : "command mode
movew -r "renumber the windows

Digital Ocean Server Setup

2017-01-22T07:20:00+03:00

Setting up a digital ocean server is pretty straight forward. First you have to pay some amount via paypal or credit card ( I paid $5). Then there is an easy html page set up tool that helps set up the account. In my case, I did not choose to use ipv6 or to use ssh key on login. However most of the options can be changed later on as I learnt. I chose Ubuntu 16.04 as my preffered OS of choice. Everything wasthen setup automatically and I just had to connect to my server.

The first thing I did was add public key authentification for my root user. I did that by:

ssh root@server-ip
answer yes to question that appears
input password

This just confirms that I can log into the system. I then exit ssh and do the following:

ssh-keygen
ssh-copy-id root@server-ip

After this process I can now login to my server without the need for my password.

I then add another user to the system who will have root privileges:

ssh root@server-ip
adduser username
usermod -aG sudo username

Add public key authentification:

ssh-keygen
ssh-copy-id username@server_ip

I then diabled password access via ssh. This makes it more secure as it prevents people guessing my server password.ie. You either have the key to access my server or not.

sudo vi /etc/ssh/sshd_config
set
    PasswordAuthentification no
sudo system reload sshd

set up firewall:

sudo ufw app list 
sudo ufw allow OpenSSH
sudo ufw status ;should see OpenSSH

I then installed nginx as my server of choice:

sudo apt update
sudo apt install nginx

And then added it to my firewall:

sudo ufw app list
sudo ufw enable 'Nginx HTTP'
sudo ufw status
systemctl status nginx

I only enable Nginx HTTP because I have not yet got an SSL cerificate, so port 43 for HTTPS is not currently in use.

This is a neat trick I got about how to get your ip address from the terminal

ip addr show eth0 | grep inet | awk '{ print $2; }' | sed
's/\/.*$//'

sudo apt-get install curl
curl -4 icanhazip.com

For a database I decided to install mariadb

sudo apt install mariadb-server mariadb-client
sudo mysql_secure_installation

I then installed php:

sudo apt-get install php-fpm php-mysql

sudo vim /etc/php/7.0/fpm/php.ini
set: cgi.fix_pathinfo =0; secure better the system

Org Mode in Vim

2017-01-09T20:20:00+03:00

So emacs has this awesome tool called orgmode that I recently saw in use. It really impressed me, especially the versatility of it. Problem is, its emacs and I'm already a vim fan boy.

So what's the next best option? Easy, look for a vim plugin, and oh boy did I get one... vim-orgmode.

It isn't perfect, heck I'd even suggest that you use the original orgmode if you really want it. But here's how I go about using orgmode:

First install it. When using vundle( I use vundle as my plugin manager), just place the line below in the .vimrc file.

Plugin 'jceb/vim-orgmode'

Then here are a few commands that will make life a breeze:

help vim-ormode "tutorial doc. You should really start here.
/hh "create a title
/cn "create a check list
/sa "create a date
/cc "mark check list as done
tab "unfold/ fold trees

So here's how I work: I create a daily document at ~/ that has the date of the current day. Something like plan_sun_05_01_2016.org Within that folder I create a tree of various plans I'd like to do Something like

* Plan for day 
** Morning tasks [/]
** Server setup [/]
** Research [/]
** Night tasks [/]

The [/] is just a way to keep a track of the subtasks on the major tasks. I then write out the subtasks. This is accomplished by \cn just below the subtask. The [/] will also be autofilled by vim. I can end up with a document that looks like this:

* Task for the day
** Morning Tasks [2/3]
   - [X] Exercise
   - [X] Eat breakfast
   - [ ] Read emails
** Server setup [0/3]
   - [ ] Buy account
   - [ ] Set up apache
   - [ ] Set up mysql
** Research [0/2]
   - [ ] How to use org mode
   - [ ] Wrtiing vim scripts
** Night tasks [0/2]
   - [ ] Read book
   - [ ] Write blog post

There are a couple of things happening above here.

The [2/3] is auto generated, and each time to add an item or mark an item with /cc it is updated.

Once I got the hang of it all, I read the docs a bit more. I got a means of adding orgmodes into the global something so that I can access their details wherever I want.

let g:org_agenda_files = ['~/org/index.org','~/org/projects.org']

Once that is set in my .vimrc, whenever I am in any org file, I can just type "\caa" to list all items for the current week and "\cat" to view all todos for the week. It just scans through the files in the list shown above.

Airtel Money and Paypal

2016-12-20T20:50:00+03:00

Note: This article is outdated. I haven't used these steps in a few years.

Paypal is an online means of paying for goods and services. It is especially useful when working with sites like ebay, where some sellers flat out only want payments through paypal.

Airtel Money is a competitor to Safaricom's Mpesa. What makes me use it is that they have the Airtel Money card, which is really easy to get and use. They are also cheaper than Mpesa. Anyway, more on that later.

So I decided I wanted to use my airtel money card with paypal. I then registered on the Paypal site. Paypal then make a withdrawal from my account. The purpose of this transaction is so as to confirm that I am the owner of the card, that is, the money transacted is not lost.

So the transaction has a description with a unique 4 digit code. The only way to access that number is from a bank statement for the card.

Unfortunately airtel money cannot provide that information. They were rather unhelpful here and I had to physically visit their stores before they gave me that information. So you have to visit Chase Bank( in Kenya). Remember to carry your National Id and the Airtel Money card. They will print for you the bank statement at no charge and give it to you.

Once you have the statement, get the unique transaction number, complete your registration on Paypal, and wallah! You can now use your card through paypal.

Remapping Keyboard Keys(Esc and Caps)

2016-12-06T20:20:00+03:00

I use vim as an editor, and this comes with some curse. I press "ESC" a lot. And yet this key is located at an inconvenient location, at the top row of my keyboard. For a key I often use, I would rather it occurs somewhere closer to the home row.

The "CapsLock" key is one that I rarely use, yet its located at the home row. Furthermore, its use is redundant because I can always use the "SHIFT" key to do the same work.

I found a nifty trick on how to remap these keys on my keyboard in linux.

Open the terminal and type:

xmodmap -e "clear Lock" -e "keycode 9 = Caps_Lock NoSymbol Caps_Lock" -e "keycode 66 = Escape NoSymbol Escape"

If this does not work replace: clear Lock with remove Lock = Caps_Lock

xmodmap -e "remove Lock = Caps_Lock" -e "keycode 9 = Caps_Lock NoSymbol Caps_Lock" -e "keycode 66 = Escape NoSymbol Escape"

However, this change is temporary. On system restart the keyboard will be the old same.

To make the changes permanent within the system:

xmodmap -pke >~/.xmodmap
# Add this line to ~/.xinitrc
xmodmap .xmodmap

Another alternative is to use setxkbmap:

# replaces caps with escape functionality
setxkbmap -option caps:escape
# swaps capslock and escape
setxkbmap -option caps:swapescape

Add whatever command to the .xinitrc file to work.

This switch is not all hymns and praises because:

My laptop just became more unusable to others
My muscle memory has already got this new configuration. This makes it difficult to work with other machines.