UP | HOME

goutils

Using the tools

  1. Run go install git.wntrmute.dev/kyle/goutils/cmd/...@latest to use the standard Go tooling.
  2. Visit the Github releases page - it uses goreleaser and some github actions to build binaries for Darwin ARM64 and Linux AMD64/ARM64.
  3. Run the docker file: docker run -it --rm kisom/goutils:latest bash — all the current tools in cmd/ are available.

Command line tools

  • atping: Automated TCP ping, meant for putting in cronjobs.
  • ca-signed: Validate whether a certificate is signed by a CA.
  • cert-bundler: Create certificate bundles from a source of PEM certificates.
  • cert-revcheck: Check whether a certificate has been revoked or is expired.
  • certchain: Display the certificate chain from a TLS connection.
  • certdump: Dump certificate information.
  • certexpiry: Print a list of certificate subjects and expiry times or warn about certificates expiring within a certain window.
  • certverify: Verify a TLS X.509 certificate file, optionally printing the time to expiry and checking for revocations.
  • clustersh: Run commands or transfer files across multiple servers via SSH.
  • cruntar: (Un)tar an archive with hard links, copying instead of linking.
  • csrpubdump: Dump the public key from an X.509 certificate request.
  • datasync: Sync the user's homedir to external storage.
  • diskimg: Write a disk image to a device.
  • dumpbytes: Dump the contents of a file as hex bytes, printing it as a Go []byte literal.
  • eig: EEPROM image generator.
  • fragment: Print a fragment of a file.
  • host: Go imlpementation of the host(1) command.
  • jlp: JSON linter/prettifier.
  • kgz: Custom gzip compressor / decompressor that handles 99% of my use cases.
  • minmax: Generate a minmax code for use in uLisp.
  • parts: Simple parts database management for my collection of electronic components.
  • pem2bin: Dump the binary body of a PEM-encoded block.
  • pembody: Print the body of a PEM certificate.
  • pemit: Dump data to a PEM file.
  • readchain: Print the common name for the certificates in a bundle.
  • renfnv: Rename a file to base32-encoded 64-bit FNV-1a hash.
  • rhash: Compute the digest of remote files.
  • rolldie: Roll some dice.
  • showimp: List the external (e.g. non-stdlib and outside the current working directory) imports for a Go file.
  • ski: Display the SKI for PEM-encoded TLS material.
  • sprox: Simple TCP proxy.
  • stealchain: Dump the verified chain from a TLS connection to a server.
  • stealchain-server: Dump the verified chain from a TLS connection from from a client.
  • subjhash: Print or match subject info from a certificate.
  • tlsinfo: Print information about a TLS connection (the TLS version and cipher suite).
  • tlskeypair: Check whether a TLS certificate and key file match.
  • utc: Convert times to UTC.
  • yamll: A small YAML linter.
  • zsearch: Search for a string in directory of gzipped files.

Libraries

  • ahash: Provides hashes from string algorithm specifiers.
  • assert: Error handling, assertion-style.
  • backoff: Implementation of an intelligent backoff strategy.
  • cache: Implementations of various caches.
    • lru: Least-recently-used cache.
    • mru: Most-recently-used cache.
  • certlib: Library for working with TLS certificates.
    • certerr: Certificate-related error handling.
    • hosts: Parsing hosts as a hostname and port.
    • pkcs7: PKCS #7 handling.
    • revoke: Revocation handling.
  • config: A simple global configuration system where configuration data is pulled from a file or an environment variable transparently.
    • iniconf: A simple INI-style configuration system.
  • dbg: A debug printer.
  • die: Death of a program.
  • fileutil: Common file functions.
  • lib: Commonly-useful functions for writing Go programs.
  • log: A syslog library.
  • logging: A logging library.
  • mwc: MultiwriteCloser implementation.
  • sbuf: A byte buffer that can be wiped.
  • seekbuf: A read-seekable byte buffer.
  • syslog: Syslog-type logging.
  • tee: Emulate tee(1)'s functionality in io.Writers.
  • testio: Various I/O utilities useful during testing.

Prompts

I used Junie to write a few of the programs.

cert-bundler

Initial

This project is an exploration into the utility of Jetbrains' Junie to write smaller but tedious programs.

Task: build a certificate bundling tool in cmd/cert-bundler. It creates archives of certificates chains.

A YAML file for this looks something like: 

config:
  hashes: bundle.sha256
  expiry: 1y
chains:
  core_certs:
    certs:
      - root: roots/core-ca.pem
        intermediates:
          - int/cca1.pem
          - int/cca2.pem
          - int/cca3.pem
      - root: roots/ssh-ca.pem
        intermediates:
          - ssh/ssh_dmz1.pem
          - ssh/ssh_internal.pem
    outputs:
      include_single: true
      include_individual: true
      manifest: true
      formats:
        - zip
        - tgz

Some requirements:

  1. First, all the certificates should be loaded.
  2. For each root, each of the indivudal intermediates should be checked to make sure they are properly signed by the root CA.
  3. The program should optionally take an expiration period (defaulting to one year), specified in config.expiration, and if any certificate is within that expiration period, a warning should be printed.
  4. If outputs.includesingle is true, all certificates under chains should be concatenated into a single file.
  5. If outputs.includeindividual is true, all certificates under chains should be included at the root level (e.g. int/cca2.pem would be cca2.pem in the archive).
  6. If bundle.manifest is true, a "MANIFEST" file is created with SHA256 sums of each file included in the archive.
  7. For each of the formats, create an archive file in the output directory (specified with -o) with that format.
    • If zip is included, create a .zip file.
    • If tgz is included, create a .tar.gz file with default compression levels.
    • All archive files should include any generated files (single and/or individual) in the top-level directory.
  8. In the output directory, create a file with the same name as config.hashes that contains the SHA256 sum of all files created.

Follow-up 1

The outputs.includesingle and outputs.includeindividual describe what should go in the final archive. If both are specified, the output archive should include both a single bundle.pem and each individual certificate, for example.

Follow-up 2

As it stands, given the following `bundle.yaml`: 

config:
  hashes: bundle.sha256
  expiry: 1y
chains:
  core_certs:
    certs:
      - root: pems/gts-r1.pem
        intermediates:
          - pems/goog-wr2.pem
        outputs:
          include_single: true
          include_individual: true
          manifest: true
          formats:
            - zip
            - tgz
      - root: pems/isrg-root-x1.pem
        intermediates:
          - pems/le-e7.pem
        outputs:
          include_single: true
          include_individual: false
          manifest: true
          formats:
            - zip
            - tgz
  google_certs:
    certs:
      - root: pems/gts-r1.pem
        intermediates:
          - pems/goog-wr2.pem
        outputs:
          include_single: true
          include_individual: false
          manifest: true
          formats:
            - tgz
  lets_encrypt:
    certs:
      - root: pems/isrg-root-x1.pem
        intermediates:
          - pems/le-e7.pem
        outputs:
          include_single: false
          include_individual: true
          manifest: false
          formats:
            - zip

The program outputs the following files:

  • bundle.sha256
  • corecerts0.tgz (contains individual certs)
  • corecerts0.zip (contains individual certs)
  • corecerts1.tgz (contains corecerts.pem)
  • corecerts1.zip (contains corecerts.pem)
  • googlecerts0.tgz
  • letsencrypt0.zip

It should output

  • bundle.sha256
  • corecerts.tgz
  • corecerts.zip
  • googlecerts.tgz
  • letsencrypt.zip

corecerts.* should contain bundle.pem and all the individual certs. There should be no \(n\) variants of archives.

Follow-up 3

Add an additional field to outputs: encoding. It should accept one of der, pem, or both. If der, certificates should be output as a .crt file containing a DER-encoded certificate. If pem, certificates should be output as a .pem file containing a PEM-encoded certificate. If both, both the .crt and .pem certificate should be included.

For example, given the previous config, if encoding is der, the googlecerts.tgz archive should contain

  • bundle.crt
  • MANIFEST

Or with letsencrypt.zip:

  • isrg-root-x1.crt
  • le-e7.crt

However, if encoding is pem, the letsencrypt.zip archive should contain:

  • isrg-root-x1.pem
  • le-e7.pem

And if it encoding is both, the letsencrypt.zip archive should contain:

  • isrg-root-x1.crt
  • isrg-root-x1.pem
  • le-e7.crt
  • le-e7.pem

Follow-up 4

The tgz format should output a `.tar.gz` file instead of a `.tgz` file.

Follow-up 5

Move the format extensions to a global variable.

Follow-up 6

Write a README.txt with a description of the bundle.yaml format.

Additionally, update the help text for the program (e.g. with `-h`) to provide the same detailed information.

Follow-up 7

It may be easier to embed the README.txt in the program on build.

Follow-up 8

For the archive (tar.gz and zip) writers, make sure errors are checked at the end, and don't just defer the close operations.