Litterate macOS configuration

This file might be most readable in an org-mode buffer directly. You can also read it from GitHub but it doesn’t show source block options like :name or :tangle.

See Read me for an introduction about this file, its goal, and how to use it. If you want to note some random “think about it later” notes, please use Roadmap or Transient scrath notes.

Deploy or retrieve configuration

If you need it you can see how to use this file.

(expand-file-name ".")

Pay special attention to any script or installation which would silently modify configuration files: it these changes are needed, replicate them here so they won’t be erased.

In order not to erase any important changes, here is how you should proceed when changing configuration:

Make sure no current running processes are likely to modify configuration processes defined in this file.
Use the script below to copy actual configuration files into this repository and see what has been changed. Commit them if needed.
Then changes everything you want, and generate tangled configuration files so you can see your changes.
Finally when you’re satisfied with your new configuration then you can copy files from the repo to $HOME. If you expect some graphical applications to take these changes into accounts, please restart them.

REPO_PATH=<<repo-path()>>
# Files related to Z-shell configuration
cp $HOME/.login  $REPO_PATH/.login
cp $HOME/.zshenv $REPO_PATH/.zshenv
cp $HOME/.zshrc  $REPO_PATH/.zshrc
cp $HOME/.zlogin $REPO_PATH/.zlogin
cp $HOME/.bashrc $REPO_PATH/.bashrc

# Files related to git configuration
cp $HOME/.gitconfig                             $REPO_PATH/.gitconfig
cp $HOME/src/github.com/.gitconfig              $REPO_PATH/src/github.com/.gitconfig
cp $HOME/src/gitlab.com/piotr-yuxuan/.gitconfig $REPO_PATH/src/gitlab.com/piotr-yuxuan/.gitconfig
cp $HOME/src/keybase/.gitconfig                 $REPO_PATH/src/keybase/.gitconfig

# Files related to ssh configuration
cp $HOME/.ssh/config $REPO_PATH/.ssh/config

# Confluent platform custom properties
mkdir -p $REPO_PATH/$(brew --prefix)/etc/ksql             && cp $(brew --prefix)/etc/ksql/ksql-server.properties                $REPO_PATH/$(brew --prefix)/etc/ksql/ksql-server.properties
mkdir -p $REPO_PATH/$(brew --prefix)/etc/kafka-rest       && cp $(brew --prefix)/etc/kafka-rest/kafka-rest.properties           $REPO_PATH/$(brew --prefix)/etc/kafka-rest/kafka-rest.properties
mkdir -p $REPO_PATH/$(brew --prefix)/etc/kafka            && cp $(brew --prefix)/etc/kafka/zookeeper.properties                 $REPO_PATH/$(brew --prefix)/etc/kafka/zookeeper.properties
mkdir -p $REPO_PATH/$(brew --prefix)/etc/schema-registry  && cp $(brew --prefix)/etc/schema-registry/schema-registry.properties $REPO_PATH/$(brew --prefix)/etc/schema-registry/schema-registry.properties
mkdir -p $REPO_PATH/$(brew --prefix)/etc/kafka            && cp $(brew --prefix)/etc/kafka/kafka.properties                     $REPO_PATH/$(brew --prefix)/etc/kafka/kafka.properties
mkdir -p $REPO_PATH/$(brew --prefix)/etc/connect          && cp $(brew --prefix)/etc/connect/connect.properties                 $REPO_PATH/$(brew --prefix)/etc/connect/connect.properties

REPO_PATH=<<repo-path()>>

# Provoke a reload or $HOME/.login
rm $HOME/.hushlogin

# Copy files from the repo to $HOME.
cp $REPO_PATH/.login  $HOME/.login
cp $REPO_PATH/.zshenv $HOME/.zshenv
cp $REPO_PATH/.zshrc  $HOME/.zshrc
cp $REPO_PATH/.zlogin $HOME/.zlogin
cp $REPO_PATH/.bashrc $HOME/.bashrc

# Files related to git configuration
cp $REPO_PATH/.gitconfig                             $HOME/.gitconfig
cp $REPO_PATH/src/github.com/.gitconfig              $HOME/src/github.com/.gitconfig
cp $REPO_PATH/src/gitlab.com/piotr-yuxuan/.gitconfig $HOME/src/gitlab.com/piotr-yuxuan/.gitconfig
cp $REPO_PATH/src/keybase/.gitconfig                 $HOME/src/keybase/.gitconfig

# Files related to ssh configuration
cp $REPO_PATH/.ssh/config $HOME/.ssh/config

# Confluent platform custom properties
cp $REPO_PATH/$(brew --prefix)/etc/ksql/ksql-server.properties                $(brew --prefix)/etc/ksql/ksql-server.properties
cp $REPO_PATH/$(brew --prefix)/etc/kafka-rest/kafka-rest.properties           $(brew --prefix)/etc/kafka-rest/kafka-rest.properties
cp $REPO_PATH/$(brew --prefix)/etc/kafka/zookeeper.properties                 $(brew --prefix)/etc/kafka/zookeeper.properties
cp $REPO_PATH/$(brew --prefix)/etc/schema-registry/schema-registry.properties $(brew --prefix)/etc/schema-registry/schema-registry.properties
cp $REPO_PATH/$(brew --prefix)/etc/kafka/kafka.properties                     $(brew --prefix)/etc/kafka/kafka.properties
cp $REPO_PATH/$(brew --prefix)/etc/connect/connect.properties                 $(brew --prefix)/etc/connect/connect.properties

Prioritary domains

Of course evaluation order matters. For example, secrets must be loaded before they are used.

Hello

Simple function providing a warm greeting towards the user.

function hello {
  echo "Hello, $USER!"
}

Managed configuration files

Here we set the header for configuration files managed from here.

We set Z-shell startup files in the same order Z-shell run them when it starts up. No previous source blocks should be tangled to these files because they would appear before the file header then, and before the shebang line.

Each of these Z-shell files is reset by Babel when tangled. As they’re to be executed by Z-shell, they start with the appropriate shabang. The first one of them is not standard in macOS, I describe it in a later section.

#!/usr/bin/env zsh
# - This file has been written automatically from configuration
#   repository <<repo-path()>>.
#   You can author this file directly, or update the script.
# - $HOME/.login is run when a user logs in and when no .hushlogin is
#   present.

The next three files are specific to Z-shell which I choose to use everywhere. Here they are presented is the order they are looked up.

#!/usr/bin/env zsh
# - This file has been written automatically from configuration
#   repository <<repo-path()>>.
#   You can author this file directly, or update the script.
# - $HOME/.zshenv is first conf file. It's run for all shells.

#!/usr/bin/env zsh
# - This file has been written automatically from configuration
#   repository <<repo-path()>>.
#   You can author this file directly, or update the script.
# - $HOME/.zshrc is run for all interactive shells, that's to say any
#   shell I can write and send commands to.

#!/usr/bin/env zsh
# - This file has been written automatically from configuration
#   repository <<repo-path()>>.
#   You can author this file directly, or update the script.
# - $HOME/.zlogin is run for all login shells, that's to say any shell
#   started as a fundamental terminal emulator interpretor (wording is
#   approximative).

#!/usr/bin/env bash
# - This file has been written automatically from configuration
#   repository <<repo-path()>>.
#   You can author this file directly, or update the script.
# - $HOME/.bashrc is a bash configuration file unrelated to zsh. It's
#   kept here because some ancillary sh scripts need some
#   configuration too.

Now we define configuration files related to git.

# - This file has been written automatically from configuration
#   repository <<repo-path()>>.
#   You can author this file directly, or update the script.
# - $HOME/.gitignore contains user-wide git
#   configuration. Configuration specific to a repository or
#   git-repository manager such as GitHub or GitLab are to be found in
#   dedicated configuration files.

# - This file has been written automatically from configuration
#   repository <<repo-path()>>.
#   You can author this file directly, or update the script.
# - $HOME/src/github.com/.gitconfig contains git configuration related
#   to git-repository manager GitHub. Configuration specific to a
#   given repository $REPO is to be found in dedicated configuration
#   files $REPO/.git/config.

# - This file has been written automatically from configuration
#   repository <<repo-path()>>.
#   You can author this file directly, or update the script.
# - $HOME/src/gitlab.com/piotr-yuxuan/.gitconfig contains git
#   configuration related to git-repository manager
#   GitHub. Configuration specific to a given repository $REPO is to
#   be found in dedicated configuration files $REPO/.git/config.

# - This file has been written automatically from configuration
#   repository /Users/p2b/environment-configuration.
#   You can author this file directly, or update the script.
# - $HOME/src/keybase/.gitconfig contains git configuration related to
#   repositories hosted on keybase. Configuration specific to a given
#   repository $REPO is to be found in dedicated configuration files
#   $REPO/.git/config.

The next header documents configuration file related to ssh

# - This file has been written automatically from configuration
#   repository <<repo-path()>>.
#   You can author this file directly, or update the script.
# - $HOME/.ssh/config is used to configure general ssh behaviour. For
#   specific configuration related to git, see the related section in
#   the configuration repository.

Locales

export LC_ALL=en_US.UTF-8

Brew path

path=('/usr/local/bin' '/usr/local/sbin' $path); export PATH

brew --prefix

Secrets in the command line

Sensitive environment variables shouldn’t be handled as shareable configuration but as secrets. In a nutshell it merely uses gpg:

SECRET_FILE=$(gmktemp)
CONFIGURATION_SECRET_KEY='<< my gpg key id >>'
echo "echo 'Secrets loaded'" > ${SECRETS_FILE}

# Encrypt your secrets
gpg --output ${SECRETS_FILE}.asc --encrypt --recipient ${CONFIGURATION_SECRET_KEY} ${SECRETS_FILE}; rm ${SECRETS_FILE}

# Source your secrets with ZSH process substitution
source <(gpg --quiet --decrypt --recipient ${CONFIGURATION_SECRET_KEY} ${SECRETS_FILE}.asc)

# Decrypt your secrets
gpg --decrypt --recipient --output ${SECRETS_FILE} ${CONFIGURATION_SECRET_KEY} ${SECRETS_FILE}.asc; rm ${SECRETS_FILE}.asc

I want to enforce some assumptions on existing environment variables.

function expect-configured-environment-variable () {
    VAR_NAME=$1
    if [[ -z "${(P)VAR_NAME}" ]]; then
        echo "Configuration error: expect environment variable $VAR_NAME to be set"
    fi
}

source <(gpg --quiet --decrypt --recipient $(cat <<repo-path()>>/.gpg-id) <<repo-path()>>/.zsh-secrets.asc)

Flat domains

Better cat and ls

brew install ls exa

alias ls=exa
alias cat='bat --paging never'

Goals

(Trust) Rely on Emacs.<<goal-0>>
(KISS principle) I want my configuration to be kept simple, stupid: most systems work better when kept simple. No need to change mindset, no need to burden beginners with something abstruse. <<goal-1>>
(principle of locality) I want a self-contained configuration in one single place. I want different settings related to the same stuff be close to each other. No other, hidden data are needed for the understanding and nobody else should produce side-effect. <<goal-2>>
(predictability) I want my command-line environment to keep the same behaviour, no matter how many times I invoke and execute this configuration file. Also, I want to be able to come back to a previous behaviour when someting goes wrong. <<goal-3>>
(declarativeness) I want to keep track of the reasonning which led to decisions and choices about this configuration. <<goal-4>>

Non goal: making this file portable and installable over the network on new computers.

It would be rather useless: I usually work on one computer everyday and don’t change that often.
It would contradict first principle. This is not an automatic installation script, write one if you want so.

Choices made to reach these goals

Rely on Emacs
- why LISP?
- why Emacs?
- Emacs is a most awesome piece of software written in a most powerful programming language. I don’t see any trouble to depend on it. On the contrary, I want Emacs to slowly infuse into my mind and to shape my thoughts.
KISS principle
- Don’t modify actual configuration without a conscious action from the user.
- User doesn’t need to be conscious about this file to modify their system configuration, they can always catch up later and see what’s changed.
- This is only a documentation, not an automatic deployment system or a configuration manager. It is what you want to do and why you chose that, no how to make it happen.
Locality
- Only use this repository for most of your general environment configuration. Structure decision by domains so different settings about the same domain end near each other. If need be, link across related concerns.
- Specialised settings related to something specific (Emacs, IntelliJ) live in their own repositories, they are imported as git modules.
- Any shell setting must originate from here. I’ve chosen to build Z-shell with option –without-etcdir so /etc/z* become muted and this goal is met (debatable, perhaps it’s less KISS).
Predictability
- Use git to version this repository so you can always checkout a prior state.
Declarativeness
- Use git to keep track of changes in time. You can see differences from the last commit as well as use git bissect when you don’t understand something weird.
- Use litterate programming to explain all the whereabouts about the configuration you choose to set up.

Learnt mistakes

`$HOME` as a git repository for configuration files

Why it’s not a good idea:

Quite a lot of programs behave differently when they are in a git repository.
It’s tedious to gitignore everything then un-ignore only specific files. Last time I checked I ended up with long, inexplicable .gitignore.
Do you feel completely quiet doing git bisect or git reset --hard on your $HOME? Why not trying git clean -x in a deep subdirectory whilst you think it’s a repository but it isn’t?

Each shell functions has its own `$HOME/bin` file

Why it’s not a good idea:

Is there any advantage doing that way in the context of a shell?
It’s a bad idea because it strongly separates the function from where it’s used.
It can help create boring rookie bugs like keychain fork bomb.

(macOS) Use `launchctl setenv` for environment variables

Why it’s not a good idea:

Most GUI apps don’t refresh their environment once they’re started.
launchtl define the environment for all macOS applications (including the terminal emulator) and then you append additional variables in your shell startup files. It’s pretty useless to expose cli variables to graphical applications.
I don’t find that kind of stuff actually fun. The proprietary Apple environment API aren’t really stable to my eyes, I don’t feel amused to waste time about them so I prefer to stay the most Unix-like possible.

If you need to tune the environment of an app, you can use plist key LSEnvironment (but leave a comment here so you will remind this in 8 months). Under the exceptional case you would need something really user-specific, you could indeed use launchctl setenv in $HOME/.login.

Use magic tools or nuclear bombs to kill a mosquito

What do you think of GitFlow? I think the basic feature branching naming is a good idea but everything else is complicated and cluttered. I always need to read a translation to raw git commands to feel confident about what’s going on. It’s a complete framework to put in your head when basic git commands are much more simple to reason about.

Perhaps you want something awfully difficult and complex so it looks more professional. Automate dotfiles deployments with Puppet or Chef if you go for it.

The ease of use is not interesting in front of the simplicity of use. Don’t automate overly, don’t add too much incidental complexity.

Put a lot of thing into `$HOME/.login` to fasten shell startup

Only put here side-effects which are to be executed once at start-up. If you want your shell to take less time to boot, choose carefully which Z-shell plugins you launch. Keep in mind that this file isn’t standard in macOS.

Spread related settings

I previously modified /etc/* and six months later I loose one full hour because of an unwise side-effect. As a result, I got the strong opinion that no side-effect should be defined out of this file. Any code, data or settings which are related one to another should be physically close or in the same location. To ensure so I compile Z-shell with option --without-etcdir <<–without-etcdir>> so it doesn’t read rc files in /etc.

How to use this file

This document merely discusses the choices made about the configuration and presents the resulting configuration files. How to bind this result to any useful configuration files is a choice left to be made to the user.

The inner repository tree structure of the tangled files matches the one of the user $HOME. That’s to say, the tangled file $REPO_PATH/.emacs.d/init.el mirrors $HOME/.emacs.d/init.el. Only exceptions: readme.org (and others like readme.pdf, etc.) and .git repository folder.

It is very very straightforward to use this file:

Evaluate code snippets with org-babel-execute-src-block.
Generate tangled files with org-babel-tangle.

Then you basically have three options:

Create symbolic links manaually
Rely on GNU Strow to manage symbolic links
Copy files from this repository onto the actual configuration files

The most simple, stupid options is to use cp so actual conf files modification requires an explicit action from the user.

Included in this file is a script to <a href=”copy files from $HOME to the repo”>copy files from $HOME to the repo. It’s useful when you want see with git diff what is about to change. Let’s define the path of this repository so we can use it later with reference extension.

You can look for read me sections for an introduction about this file, its goal, and how to use it.

Quick introduction on org-mode and Babel

Thorough descriptions of org-mode and Babel are available on their respective manual.

In a very simple way, this file contains source blocks. Emacs, when asked properly (thanks to Babel), can write these blocks out to specified files. For example, the following block would be written out to ./my-file.txt – this file is said to be tangled.

#+BEGIN_SRC text :tangle ./my-file.txt
  This is the new content of file
#+END_SRC

You can also execute code in source blocks like the following one. Here I specify shell as the source language so it will get executed ith my default shell (which is Z-shell).

#+BEGIN_SRC shell :results silent
  cp ./my-file.txt ./renamed-file.txt
#+END_SRC

Finally, let’s use block arguments and reference expension. Arguments are passed to the source block as they would be set in the language. Reference expension are a bit like macro and get replaced in the body of the block before it gets evaluated. This last block is equivalent to the previous one.

#+NAME: current-file
#+BEGIN_SRC emacs-lisp
  (concat "my-file" ".txt")
#+END_SRC

#+BEGIN_SRC shell :results silent :var CURRENT_PATH=(expand-file-name ".")
  cp ./<<current-file()>> $CURRENT_PATH/renamed-file.txt
#+END_SRC

Yeah, I know, Emacs is powerful and can turn a litterate description of your settings into a effectful REPL.

Git, the version control software

expect-configured-environment-variable GITHUB_TOKEN_CLI

[includeIf "gitdir:~/src/github.com/**"]
    path = ~/src/github.com/.gitconfig
[includeIf "gitdir:~/src/gitlab.com/piotr-yuxuan/**"]
    path = ~/src/gitlab.com/piotr-yuxuan/.gitconfig
[includeIf "gitdir:~/src/keybase/**"]
    path = ~/src/keybase/.gitconfig

[gpg]
    program = /usr/local/bin/gpg
[commit]
    gpgsign = true
[gc "refs/remotes/*"]
    reflogExpire = 35 days
    reflogExpireUnreachable = 35 days
[core]
    excludesfile = ~/.gitignore

Tags in this file

Read me

Sections with this tag describe this file and how to get acquainted with it. Click on the tag to see all sections which bear it.

Text or code editor

Programming language

This current local machine

macOS configuration

Shell, or terminal configuration

Small, minor tool which can help

Day to day tasks to keep your system up to date : bump versions, update declarative versions, synchronise settings.

Organisation of `$HOME`

/Users/$USER
├── .emacs.d/
│   ├── init.el (generated by init.org)
│   └── init.org (litterate init.el)
├── Desktop
├── bin/ (binary files or executable scripts)
├── dist/ -> $HOME/.m2/repository/
├── environment-configuration/
│   └── readme.org (this file)
├── img/ -> Pictures/
│   ├── screenshots/ (where screenshots are put)
│   └── pvt -> $HOME/pvt/img/
├── lib/ -> Library/
├── man/c
│   └── pvt -> $HOME/pvt/man/
├── mov/ -> Movies/
│   └── pvt -> $HOME/pvt/mov/
├── net/ -> Downloads/
│   └── pvt -> $HOME/pvt/net/
├── pkg/
├── pvt -> Documents
│   ├── img/
│   │   └── library -> $HOME/pvt/library/
│   ├── library/ (enclosed library files, flat directory)
│   ├── man/
│   ├── mov/
│   │   └── library -> $HOME/pvt/library/
│   ├── net/
│   └── snd/
│       ├── music/ (flat directory or music I listen to)
│       └── library -> $HOME/pvt/library/
├── snd -> Music/
│   └── pvt -> $HOME/pvt/snd
└── src/ (for source code)
    ├── github.com/ (host/username/repo)
    └── …

I feel like it would really be a terrible idea to actually rename user folders like Documents and Movies because there are very standard folders which are not meant to change. Even with a standard macOS tool to say “OK, now let’s change the default folder for pictures from Pictures to img”, I can’t guarantee that no program wouldn’t blindly assume it exists.

I have chosen to hide default folders and create symbolic links to them so it looks like what I want but the change doesn’t bring too deep implication and weird bugs.

These file names are inspired from what golang expects.

`$HOME/.login`

$HOME/.login isn’t a standard file in macOS. However, I’ve found it a nice tool for side-effects which must be invoked once in a while and not for each new shell (for example ssh configuration).

Custom shell prompts when you first open a terminal in a session. You wanna trigger it only once after your login or $HOME.login is reloaded.

# If it's in a terminal and no .hushlogin is present
if [[ ( -t 1 ) && ( ! -f $HOME/.hushlogin ) ]]; then

    source $HOME/.login
    touch $HOME/.hushlogin # Don't execute it again

    archey 2>& /dev/null
    echo "$(fortune)\n"
fi

Also, you still need to remove hushlogin on startup.

Important : this file can’t contain variable like $USER, change it to what you need to explicit absolute path.

;; $HOME/Library/LaunchAgents/user.login.plist

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
	  <dict>
		  <key>Label</key>
		  <string>user.login</string>

		  <key>ProgramArguments</key>
		  <array>
			  <string>/bin/rm</string>
			  <string>$USER/.hushlogin</string> <!-- FIXME -->
		  </array>

		  <key>RunAtLoad</key>
		  <true/>

		  <key>WatchPaths</key>
		  <array>
			  <string>$HOME/.login</string>
		  </array>
	  </dict>
</plist>

This file is not included in the source block which copy files to $HOME because it is almost never updated. In addition to that, it must be reloaded with the following script whenever it changes.

launchctl unload -w $HOME/Library/LaunchAgents/user.login.plist
launchctl load -w $HOME/Library/LaunchAgents/user.login.plist

How to set `$PATH`

How to set $PATH in Z-shell https://stackoverflow.com/a/18077919.

Here I choose to set $PATH within context. That’s to say, when I need to append something to $PATH because I’m setting up a program, I do it in the context of this program. I always do it from this file. As a result, I can always easily find where $PATH is set (it originates from this file) and why it’s been set up this way.

I have previously gone great length to use path_helper. I found its behaviour to be hard to predict so I even implemented it again in a simple zsh script which read .path as well as .path.d/*. I’m no longer amused with that kind of accidental complexity and now I prefer to set path here in this file so I know the context.

Hostname

https://apple.stackexchange.com/a/90873

sudo scutil --set ComputerName $hostname
sudo scutil --set LocalHostName $hostname
sudo scutil --set HostName $hostname

dscacheutil -flushcache
# Perhaps you will need to restart applications or even you computer
# for this setting to take effect.

Brew, macOS package manager

expect-configured-environment-variable HOMEBREW_GITHUB_API_TOKEN

Keeping your system up to date.

After this page. It would be wonderful to use mas. Err, wait, not really. I use almost no program from app store except Line. If I use Line, I must be sure I can avoid OS upgrades.

When running this script with greedy cask update, better to run this snippet in the CLI as your password might be asked for.

brew update
brew upgrade
brew cask upgrade --greedy # greedy reinstall latest version of
			   # unversioned casks. Suboptimised, takes
			   # significantly more time.
brew cleanup -s
brew cask cleanup
serialize_brew_state
echo "you should commit your environment configuration"

Reproducible environment

How to get a reproducible configuration with brew? I will later use Nix, the functional package manager. For now I can at least get a description of installed software with brew.

function serialize_brew_state {
  REPO_PATH=<<repo-path()>>

  # We don't care about actual Brewfile because it will be regenerated
  # soon and it's versionned anyway.
  rm $REPO_PATH/Brewfile

  # Generate Brewfile with hard location
  pushd $REPO_PATH
  BREWFILE_PATH=. brew bundle dump
  popd

  # List installed software with version
  brew list --versions > $REPO_PATH/brew-list-versions

  # List installed software with version (cask)
  brew cask list --versions > $REPO_PATH/brew-cask-list-versions
}

Change current directory to frontmost window of macOS Finder.

pfd () {
    currFolderPath=$( /usr/bin/osascript <<EOT
	  tell application "Finder"
	      try
	  set currFolder to (folder of the front window as alias)
	      on error
	  set currFolder to (path to desktop folder as alias)
	      end try
	      POSIX path of currFolder
	  end tell
EOT
    )
    "$currFolderPath"
}

Recursively delete .DS_Store files

alias cleanupDS="find . -type f -name '*.DS_Store' -ls -delete"

Hide or show hidden files in Finder

alias finderShowHidden='defaults write com.apple.finder AppleShowAllFiles TRUE'
alias finderHideHidden='defaults write com.apple.finder AppleShowAllFiles FALSE'

Compile Z-shell files (zcompile)

It looks tempting to compile Z-shell files in an attempt to boost your start up time.

https://github.com/antonio/zsh-config/blob/285f5ac6955c19908c30935b312d3521d0e0b5e2/help/zcompile

However it appears not to be a good idea because whenever a script silently try to append anything into a Z-shell configuration files (for example, an installation script add something in $PATH) it creates a new file which has precedence over compiled file (because of a more recent timestamp).

Keep it simple: don’t use compile startup files so you can modify your Z-shell files without much thinking about that.

Network helpers

alias myip='curl ip.appspot.com'                    # myip:         Public facing IP Address
alias netCons='lsof -i'                             # netCons:      Show all open TCP/IP sockets
alias flushDNS='dscacheutil -flushcache'            # flushDNS:     Flush out the DNS Cache
alias lsock='sudo /usr/sbin/lsof -i -P'             # lsock:        Display open sockets
alias lsockU='sudo /usr/sbin/lsof -nP | grep UDP'   # lsockU:       Display only open UDP sockets
alias lsockT='sudo /usr/sbin/lsof -nP | grep TCP'   # lsockT:       Display only open TCP sockets
alias ipInfo0='ipconfig getpacket en0'              # ipInfo0:      Get info on connections for en0
alias ipInfo1='ipconfig getpacket en1'              # ipInfo1:      Get info on connections for en1
alias openPorts='sudo lsof -i | grep LISTEN'        # openPorts:    All listening connections
alias showBlocked='sudo ipfw list'                  # showBlocked:  All ipfw rules inc/ blocked IPs

Syntactic sugar for search

alias f="find . -name "                     # f:        Quickly search for file
ff () { /usr/bin/find . -name "$@" 2>/dev/null ; }      # ff:       Find file under the current directory
ffs () { /usr/bin/find . -name "$@"'*' 2>/dev/null ; }  # ffs:      Find file whose name starts with a given string
ffe () { /usr/bin/find . -name '*'"$@" 2>/dev/null ; }  # ffe:      Find file whose name ends with a given string

iTerm2

I use iTerm2 as a terminal but you could use built-in Apple Terminal. I use Z-shell as a shell. Recently I’ve been trying extraterm.

Installation

brew cask info iterm2-nightly

Configuration

Configuration can be exported into json.

Integration

path+=("$HOME/.iTerm2"); export PATH

Z-shell

Installation

See Brewfile for compilation switch. –without-etcdir is enabled: it’s a custom setting so you have to know it but it removes even the possibility that data in etc dir change the behaviour of zsh, which can be quite painful to debug.

brew install zsh --without-etcdir --with-unicode9 --with-pcre

Define Z-shell as you default session shell:

chsh -s $(which zsh)

I install the latest (stable) version of Z-shell. I compile it with an option which explicitly disables the reading of Zsh rc files in /etc <<without-etcdir>>, so it further enforces that all configuration must come from here.

TODO zsh opts?

Basic configuration

This obviously targets interactive shells.

export VISUAL='emacsclient --create-frame'
export EDITOR='emacsclient'
export ARCHFLAGS="-arch x86_64"

I need history variables for interactive shells, they would be useless elsewhere.

export HISTSIZE=10000000
export HISTFILESIZE=100000000
export HISTTIMEFORMAT="%d/%m/%y %T "

# https://leetschau.github.io/remove-duplicate-zsh-history.html
setopt EXTENDED_HISTORY          # Write the history file in the ":start:elapsed;command" format.
setopt INC_APPEND_HISTORY        # Write to the history file immediately, not when the shell exits.
setopt SHARE_HISTORY             # Share history between all sessions.
setopt HIST_EXPIRE_DUPS_FIRST    # Expire duplicate entries first when trimming history.
setopt HIST_IGNORE_DUPS          # Don't record an entry that was just recorded again.
setopt HIST_IGNORE_ALL_DUPS      # Delete old recorded entry if new entry is a duplicate.
setopt HIST_FIND_NO_DUPS         # Do not display a line previously found.
setopt HIST_IGNORE_SPACE         # Don't record an entry starting with a space.
setopt HIST_SAVE_NO_DUPS         # Don't write duplicate entries in the history file.
setopt HIST_REDUCE_BLANKS        # Remove superfluous blanks before recording entry.

Oh-my-zsh

I would clearly prefer Z-plug here. I will use it when I’ll have time.

 export ZSH=$HOME/.oh-my-zsh

 # I should better use TERM_PROGRAM
 case $TERM in
     # - Emacs term and multi-term
     eterm-color) export ZSH_THEME=lambda
		   ;;
     # - Emacs eshell
	# not a zsh shell, don't read this file
     # - iTerm
     # - Terminal (macOS standard app set up for this)
     xterm-256color) export BULLETTRAIN_DIR_EXTENDED=2
		      export BULLETTRAIN_PROMPT_ADD_NEWLINE=false
		      export BULLETTRAIN_PROMPT_ORDER=(git context dir time)
		      export ZSH_THEME=bullet-train
		      ;;
 esac

 # Yes, I'm a sinner. (enables oh-my-zsh auto updates)
 export DISABLE_UPDATE_PROMPT=false

 # zsh-autosuggestions git git-extras emacs aws npm node go golang lein
 plugins=(git git-extras emacs go golang lein zsh-iterm-touchbar)
 source $ZSH/oh-my-zsh.sh

 if [[ 'iTerm.app' = $TERM_PROGRAM ]]; then
   source "${HOME}/.iterm2_shell_integration.zsh"
 fi

Firefox dev edition

I’ve modified chrome/userChrome.css to remove the “red dot” on new tab title change, which Firefox makes a less aggressive blue light. Actually, even if not that aggressive I don’t want to be disturbed by Twitter and Facebook.

.tabbrowser-tab > .tab-stack > .tab-content[pinned][titlechanged] {
  background-image: none !important;
}

The idea comes from this answer.

SSH configuration

Only express here configuration related to ssh connection to some actual host through ssh. For git ssh keys, see the git config which gracefully handles multiplie identities.

# Add these keys once and for all
ssh-add -K $HOME/.ssh/mine.pem

Host *
  UseKeychain           yes
  AddKeysToAgent        yes

Host github.com
  HostName              github.com
  User                  git

Host gitlab.com
  HostName              gitlab.com
  User                  git

Shell utilities

GPG Terminal

export GPG_TTY=$(tty)

Jump over directories

I use z for it. It’s like j but if I remember well it’s in pure script.

source <<brew-prefix()>>/etc/profile.d/z.sh

I also define some shortcuts. They are to be used only by myself directly from the command line, hence I put them in ~./.zshrc.

export SRC="$HOME/src"
export GH="$SRC/github.com"
export GL="$SRC/gitlab.com"
export SND="$HOME/snd"
export PVT="$HOME/pvt"
export MOV="$HOME/mov"
export NET="$HOME/net"

Colourful man pages

This come from boredzo.org.

man() {
    env \
	  LESS_TERMCAP_mb=$(printf "\e[1;31m") \
	  LESS_TERMCAP_md=$(printf "\e[1;31m") \
	  LESS_TERMCAP_me=$(printf "\e[0m") \
	  LESS_TERMCAP_se=$(printf "\e[0m") \
	  LESS_TERMCAP_so=$(printf "\e[1;44;33m") \
	  LESS_TERMCAP_ue=$(printf "\e[0m") \
	  LESS_TERMCAP_us=$(printf "\e[1;32m") \
	  man "$@"
}

Better htop: gtop

https://github.com/aksakalli/gtop

Regular expression

Syntactic sugar for using regular expression with AWK.

function regex {
  gawk 'match($0,/'$1'/, ary) {print ary['${2:-'0'}']}'
}

Feedback sounds

Put any sound you like. It helps making human-computer interactions more human.

function yay {
  afplay $HOME/snd/yay.m4a > /dev/null 2>&1 &
}

function mwahaha {
  afplay $HOME/snd/mwahaha.mp3 > /dev/null 2>&1 &
}

Generate a gif from a screen record

I’ve always thought a fully fledged screen record is way too heavy when you just want to report basic interaction.

function gifgo {
  ffmpeg -i "$1" -pix_fmt rgb24 -r 10 -s 900x600 -f gif - | gifsicle --delay=15 --optimize=3 > "$1.gif"
}

Generate and display a QR code from the clipboard

A space is prepended before the first argument just for my own convenienve: if not, links are parsed as links by my phone and not as text (I prefer text).

function qrgo {
    tmp_file=$(mktemp).png
    qrencode  -o - "$(pbpaste)" > $tmp_file
    open $tmp_file
}

Grep json

brew install gron

Example of how to use it:

gron "https://api.github.com/repos/tomnomnom/gron/commits?per_page=1" | fgrep "commit.author" | gron --ungron

It’s like jq but in less powerful but easier to use: gron’s primary purpose is to make it easy to find the path to a value in a deeply nested JSON blob when you don’t already know the structure; much of jq’s power is unlocked only once you know that structure.

Query json with jq

brew install jq

Here is the documentation.

Query csv or tsv with sql

brew install q

https://github.com/harelba/q

q "SELECT COUNT(*) FROM ./clicks_file.csv WHERE c3 > 32.3"

ps -ef | q -H "SELECT UID,COUNT(*) cnt FROM - GROUP BY UID ORDER BY cnt DESC LIMIT 3"

fzf, command-line fuzzy finder

https://github.com/junegunn/fzf

[ -f ~/.fzf.zsh ] && source ~/.fzf.zsh

C-t Paste the selected files and directories onto the command-line
C-r Paste the selected command from history onto the command-line (C-r again for chronological order)
M-c Move into the selected directory

Python

I may have installed Python but I’ve quite seldom used it. However the warning message looks like something which is awfully boring to debug so I put this warning here as it will help me in six months.

$ brew doctor

Warning: Putting non-prefixed findutils in your path can cause python builds to fail.

path+=('/usr/local/opt/python/libexec/bin'); export PATH

Haskell

path+=("$HOME/.cabal/bin"); export PATH

Go language

See what I put into my =HOME=.

export GOPATH="$HOME/go"

Lilypond

I put it in $HOME/.zshenv because it’s fair enough to consider lilypond as a command. As so, it should be widely available.

function lilypond {
  /Applications/LilyPond.app/Contents/Resources/bin/lilypond "$@"
}

Rust

path+=("$HOME/.cargo/bin"); export PATH
rustc --print sysroot

path+=("$HOME/.cargo/bin"); export PATH
export RUST_SRC_PATH=<<rust-source-path()>>/lib/rustlib/src/rust/src

Count line of codes in a directory

cargo install loc

Clojure

Clojure can be installed through brew. Leiningen settings (in $HOME/.lein) are in its own separate repository.

Custom Maven location

Maven location should not be custom. However I feel like it’s more explicit to set it here so you have no doubt on my intention (look, I’m honest, I’m setting it here ^^).

export M2_HOME=$HOME/.m2/repository # Default value, made explicit

Confluent platform

I’ve installed confluent-oss.

Default port configuration for localhost.

Component	Port
Apache Kafka brokers (plain text)	9092
Confluent Control Center	9021
Kafka Connect REST API	8083
REST Proxy	8082
Schema Registry REST API	8081
ZooKeeper	2181

Install confluent platform with:

brew install confuent-platform

brew --prefix confluent-platform

export CONFLUENT_HOME=<<confluent-home()>>

According to the documentation I install the command line can be installed with:

curl -L https://cnfl.io/cli | sh -s -- -b $(brew --prefix)/bin

Start the platform with

confluent local start

After reading https://github.com/confluentinc/confluent-cli I feel it could be better to add confluent home in my env vars.

Android

According to getting started of React native:

export ANDROID_HOME=$HOME/Library/Android/sdk
export PATH=$PATH:$ANDROID_HOME/tools
export PATH=$PATH:$ANDROID_HOME/tools/bin
export PATH=$PATH:$ANDROID_HOME/platform-tools
expect-configured-environment-variable PIOTR_YUXUAN_GRADLE_SIGNING_KEY_PASSWORD

Some scripts also needs these data but run in bash.

export ANDROID_HOME=$HOME/Library/Android/sdk
export PATH=$PATH:$ANDROID_HOME/tools
export PATH=$PATH:$ANDROID_HOME/tools/bin
export PATH=$PATH:$ANDROID_HOME/platform-tools
expect-configured-environment-variable PIOTR_YUXUAN_GRADLE_SIGNING_KEY_PASSWORD

Reproductible configuration shows the version of Android Studio I run.

PostgreSQL

Command line tools are made available with:

path+=('/usr/local/opt/postgresql/bin'); export PATH

CLI tools

export PGDATA="/usr/local/var/postgres"

Socket error, can’t connect

If PostgreSQL looks started in brew services list but you can’t acquire a connection to its socket, perhaps it’s because you’ve had a shutdown which wasn’t clear for postgre. In this case, do the following:

rm /usr/local/var/postgresql/postmaster.pid
brew services restart postgresql

Cassandra, C*

https://gist.github.com/Micka33/89897e1490240a56c036

Properties: /usr/local/etc/cassandra Logs: /usr/local/var/log/cassandra Data: /usr/local/var/lib/cassandra/data

Schemata, schema registry

export SCHEMA_REGISTRY_URL=http://localhost:8081
export SCHEMA_REGISTRY_URI=http://localhost:8081

Copy or reset schemas topic in local

function destructive_reset_local_schemas_to_latest_production {
  confluent local destroy
  echo "\$(confluent local current | tail -n1)=$(confluent local current | tail -n1)"
  confluent local start
  echo 'Kafka topics after $(confluent local start):'
  kafka-topics \
  --zookeeper localhost:2181 \
  --list
  echo 'Creating _schemas-test'
  kafka-topics \
    --zookeeper localhost:2181 \
    --create \
    --topic _schemas-test \
    --partitions 1 \
    --replication-factor 1 \
    --config cleanup.policy=compact
  DUMP_FILE=$(gmktemp -d)/_schemas_$(date +%Y%m%d%H%M).log
  echo "DUMP_FILE=$DUMP_FILE"
  echo "Are you using the VPN?\n"
  echo "Consume production schema topic"
  kafka-console-consumer \
    --bootstrap-server bootstrap.server:9092 \
    --topic _schemas \
    --from-beginning \
    --property print.key=true \
    --timeout-ms 15000 \
    1> $DUMP_FILE
  echo "kafka.consumer.ConsumerTimeoutException is fine: it means the topic"
  echo "has been read to the end and no more message has been found, hence"
  echo "causing the timeout.\n"
  echo "Produce messages to local topic _schemas-test"
  cat $DUMP_FILE | kafka-console-producer --broker-list localhost:9092 --topic _schemas-test --property parse.key=true
  echo "Restart schema registry"
  confluent local stop schema-registry
  confluent local start schema-registry
}

Amazon environment variables

expect-configured-environment-variable AWS_ACCESS_KEY_ID
expect-configured-environment-variable AWS_SECRET_ACCESS_KEY

IntelliJ settings repository

Token for IntelliJ to manage its settings repository.

expect-configured-environment-variable INTELLI_J_SETTINGS_REPOSITORY

Reduce the size of my music library

Here is how to extract audio for a single file and

if ffmpeg -i a.mkv -vn -acodec copy a.ogg ; then
    rm a.mkv
else
    echo "Failed for a.mkv"
fi

The result is:

name	size
a.mkv	7.8M
a.ogg	2.0M

So I’m pretty convinced. Moreover, according to Wikipedia, « being a container format, Ogg can embed audio and video in various formats » so I guess it doesn’t lower quality too much. I’ve quickly checked with Audacity: audio rate is at 48k so it’s nice.

Let’s extract audio for all files. I want to keep things simple and use two functions: one to search for files, and another one to convert them. However it’s hitting some limitations of find and sub shell spawning (which doesn’t import functions) so let’s do it in a dirty way.

 find . -type f -name "*.mkv" -exec bash -c 'FILE="$1";
   if ffmpeg -i "$FILE" -vn -acodec copy "$FILE.ogg" ; then
	rm "$FILE"
   else
	echo "Failed for $FILE"
   fi' _ '{}' \;

Before I actually launch it on my music files, let’s try it. Pick a bunch of random files, copy them into a folder in /tmp and record how much they weight. dh -sh . returns 557M. Note that it includes also non-mkv files. The actual mkv files are 453M.

Now let’s run the little script. The ogg replacement are 145M. Quite good, we’ev got a 66% size reduction! The other files are untouched. OK, it passes the test, let’s release it.

The initial size of ~/Documents/snd/music is 179G, that’s big! If the same ratio of 66 is kept, I expect a minimal final size of 60G (just like all files would be mkv).

Oups, I’ve just thought I could have some actual videos in this music, for example a ballet music. Well, I guess it’ll be only a few of them, and music will remain. I don’t often change names, and YouTube video names often contain a code to uniquely identify the video. It should be enough to recover the few I could actually badly miss.

Now it’s around 98G after this first command has completed. We’ve already freed 81G for binaries or code! Now the free space ratio of my mac 1T drive is around 28%. I guess it’s enough for now. Next time I’ll try to compress other file formats.

OpenJDK in spite of Oracle JDK

No joke, bro.

export OPEN_JDK_HOME="$HOME/.jenv/bin"
path+=($OPEN_JDK_HOME); export PATH

eval "$(jenv init -)"

Name		Name	Last commit message	Last commit date
Latest commit History 2 Commits
.gitignore		.gitignore
LICENSE		LICENSE
readme.org		readme.org

License

piotr-yuxuan/public-environment-configuration

Folders and files

Latest commit

History

Repository files navigation

Litterate macOS configuration

Table of contents

Deploy or retrieve configuration

Prioritary domains

Hello

Managed configuration files

Locales

Brew path

Secrets in the command line

Flat domains

Better cat and ls

Goals

Choices made to reach these goals

Learnt mistakes

$HOME as a git repository for configuration files

Each shell functions has its own $HOME/bin file

(macOS) Use launchctl setenv for environment variables

Use magic tools or nuclear bombs to kill a mosquito

Put a lot of thing into $HOME/.login to fasten shell startup

Spread related settings

How to use this file

Quick introduction on org-mode and Babel

Git, the version control software

Tags in this file

Read me

Organisation of $HOME

$HOME/.login

How to set $PATH

Hostname

Brew, macOS package manager

Keeping your system up to date.

Reproducible environment

Change current directory to frontmost window of macOS Finder.

Recursively delete .DS_Store files

Hide or show hidden files in Finder

Compile Z-shell files (zcompile)

Network helpers

Syntactic sugar for search

iTerm2

Installation

Configuration

Integration

Z-shell

Installation

Basic configuration

Oh-my-zsh

Firefox dev edition

SSH configuration

Shell utilities

GPG Terminal

Jump over directories

Colourful man pages

Better htop: gtop

Regular expression

Feedback sounds

Generate a gif from a screen record

Generate and display a QR code from the clipboard

Grep json

Query json with jq

Query csv or tsv with sql

fzf, command-line fuzzy finder

Python

Haskell

Go language

Lilypond

Rust

Count line of codes in a directory

Clojure

Custom Maven location

Confluent platform

Android

PostgreSQL

CLI tools

Socket error, can’t connect

`$HOME` as a git repository for configuration files

Each shell functions has its own `$HOME/bin` file

(macOS) Use `launchctl setenv` for environment variables

Put a lot of thing into `$HOME/.login` to fasten shell startup

Organisation of `$HOME`

`$HOME/.login`

How to set `$PATH`