Kibihrchak’s Best Webpage

VS Code Remote Development

2022-01-10T00:00:00+00:00

When remote work is not enough.

Initial Encounter with VS Code Remote Development

During the past week I gave VS Code Remote Development a try, mainly to see how it would work with a build environment set up in a Docker container. Victim for this was this blog repo, so the work went in setting up an image with Jekyll/GitHub Pages, and feeding it this repo’s content to inspect the locally generated output.

First try went with using the Docker Desktop on the Windows host. But, it appeared that running Docker Desktop relies on Hyper-V being enabled, and that brought its own set of issues, in particular with VirtualBox. With that encountered, I’ve moved the Docker installation to the VirtualBox Ubuntu VM with an idea to run VS Code there (better something than nothing). But, as it happened, VS Code offers an option to remotely connect to a container through SSH. This allowed me at the end to have the following setup which surprisingly enough works without a hitch:

VS Code runs on the Windows host.
It can SSH to the Ubuntu VirtualBox VM and open up workspaces there.
Furthermore, it can run a Docker container in the said VM and work on a workspace inside of it.

Common Development Interface

The most significant benefit of this setup for me is the ability to use the common development interface, but tailored to the peculiarities of the particular project. This kind of setup before required either -

Doing the development on the same base system, which led to the overlapping of development environment setups (needed software, VS Code extensions), or
Having multiple instances of VS Code deployed in independent development environments, which then required repeated base setup of VS Code, and introduced an effort for keeping this base setup same between all environments.

With this remote development setup, it is possible to use the same, host-based instance of VS Code, and then have separate configurations for each independent development environment. First point allows for the speed and convenience of doing the work from the host-based interface, and not having to run the VS Code through virtual machine instances. Second point means that the development environments are not mashed together which complicates provisioning and gives opportunity for environment collisions.

In this vein, an ideal development setup for me should be looking something like this:

Have the base VS Code setup on the host system for host-related projects (that is, text-based projects, or the software projects intended to be ran on the host system).
For each non-host project, their build/test environment is set up in an isolated fashion through Docker.

Mobile Phone Cameras

2021-11-13T00:00:00+00:00

The best camera mode is the one you have on your phone.

Leveraging Computational Photography

Some day ago I’ve listened to the MKBHD’s podcast on Pixel 6 and heard about the new photo modes introduced in this new lineup of Pixel phones. That got me thinking on the overall development of mobile photography, and how it managed to improve and get to the state where it is today.

I mean, the mobile phone photography compared to the conventional one is at a clear disadvantage when compared to the conventional one due to the physical limitations on what you can fit in the mobile phone form factor. This results in mobile phones having a smaller sensor size and smaller equivalent aperture, and consequently resulting in photos lacking depth of field and being noisier when compared to the conventional camera counterpart ones, in addition to being limited to a single focal length (there’s a great video explaining the senosr size differences by Tony & Chelsea Northrup).

So, what changed here that allowed us to move from potato cell phone photos to what we have today? Sensor size and resolution increased during time, alright, but aside from that two other things occured. First one was brute-forcing the way around the fixed focal length limitation by introducing multi-camera setup with each camera having its own optics stack. But, what is more important to me here is the other approach to achieving better photos, and that is through the computational photography (CP), the ability to infer more information about the photo and then improve it, either by computer vision, image stacking, or sensor fusion.

Here the current application examples that come to my mind are:

The subject isolation, basically emulating depth of field of larger lenses by blurring the “out-of-focus” image areas either by recognizing the subject on the image, or using LIDAR to create a scene depth map,
HDR, stacking different exposure photos of the same scene to achieve the greater dynamic range,
Night and astrophotography modes as a follow-up to the HDR,
Automated panoramic stitching, although in the recent years ultrawide camera setups replaced this mode for getting the wide lens camera shots,
Improved digital zoom, where instead of having only one photo, multiple photos are stacked to resolve additional detail from digitally zoomed photo,
Most recently, Pixel 6 introduced motion photo modes for getting the panning and long exposure shots.

Camera as a Code

One important aspect of the CP is that the resulting image content is not determined anymore only by the raw captured data, but by what can be extracted out of it, and what can be retrieved from other photos and other sensors. Thus, it is subject to change as the processing mechanisms gets updated, allowing for better photos or even further modes.

Ofc, the limiting factor here are the phone processing capabilities, because if image processing takes too long it would deteriorate the user experience. That’s the reason that new features require custom hardware to keep them going.

Computational Photography is Cheating

There’s one more thing to consider here, and that is that the CP can be seen as a “cheating” when compared to the conventional photography, basically producing image information that didn’t exist in the original photo. The counter-argument here is that indeed the photo coming out of a compact or DSLR was of better quality and could be used straightaway, but still for getting the most of the photo some processing in post was needed. That could have been eg. stitching the photos to get the panoramic shot unattainable by the used lens setup, or pulling in shadows/highlights to get a HDR look, or even retouching the photos to remove the undesirable elements from the photo.

Indeed, these operations as well as getting some kind of shots, like action pan ones require more skill and effort and thus can be more valued (putting your sweat into the photo), but from the technical standpoint they differ not that much from what CP achieves, the only difference being that these operations are done by hand and not by an automated system.

Where Do We Go Next?

The initial incentive for CP came out of overcoming the limitations of mobile phone camera setups, and trying to achieve something what could be got from a conventional camera. I think that for consumer use we’ve reached the stage where even the mid-range smartphones offer the functionality set that covers most of the use cases and that making progress there becomes less important.

One non-CP direction would be to simplify the camera setup and try to revert to a single-camera configuration with the aim to reduce the cost and free up phone real estate. The most notable technologies there are -

Using the periscope camera setup allowing for horizontal stacking of optic elements, and thus allowing longer focal lengths and even adjusting the focal length, and
Use of liquid lenses.

As for the CP, the next step now could be assisting the photographer in improving the photo content, either passively (composition assistance, selecting the best shot out of the burst) or actively (image retouching). Some examples would be -

Pixel 6 motion photo modes,
Pixel 6 in-phone unwanted object removal,
Galaxy S10 best shot suggestion.

For now, these modes seem like a manufacturer’s gimmick made for the phone to stand out among the competition, but like with HDR or night mode or the rest, I would assume the proliferation into general use when the technology settles in and the supporting hardware becomes more widespread and universal and not vendor-specific.

Autonomous Driving Observations

2021-10-30T00:00:00+00:00

“Tap on the stars to rate your autonomous ride - ⭐⭐⭐⭐⭐”

Driving Instructor to an Autonomous Driving System

While watching the Tesla FSD videos from AI DRIVR, one thing about autonomous driving caught my attention. My previous perception about autonomous driving and its categorization was that -

Either car has full control over the driving and requires drivers assistance when it cannot determine the correct course of action, or
That car automates only certain aspect of driving (eg. cruise control), and leaves the rest to the human driver.

But one additional aspect that appeared really significant to me there was having the ability to act like a driving instructor to the autonomous driving system. This occured on several of the Tesla FSD videos, where the human driver didn’t take control over the car from the autonomous driving system, but rather gave it a hint what it could do. Some examples seen there were - giving it a nudge when being stopped at the intersection that it is safe to advance, or requesting the lane change when the car seems unsure about doing it. Other things that come to my mind and follow this suit would be keeping the steering wheel while turning and thus saying to the car that it is turning too sharply, or tapping the brake to tell the car to slow down, or tapping the accelerator to tell the car that it is okay to take over another vehicle.

This whole “driving instructor” functionality to me seems like a really good approach to bridge the gap while developing an autonomous system of level 2 and above. The important thing is that the system still has the ability to get the vehicle to a safe state if it is unable to perform anymore, but the instructor thing allows for a partially done system to still be usable. Then, with time as the system improves, driver’s assistance would be required less and less. And, about the improvement, this assistance aspect has a real value if the driving system behavior is monitored and that data aggregated to improve the system.

“It is Human to Make Mistakes”

Another noticeable thing while watching the videos, and something that AI DRIVR has also mentioned on a couple of occassions is the ability of the autonomous driving system to gracefully deal with mistakes. It is one thing to detect that you in a mess and just freeze there and yepl for hooman’s assistance, and other thing is to try to make things better. Some examples of that would be -

Trying to move to the side of the road if there’s not enough room for the vehicle from the opposite direction to pass.
Trying to claw your way through an intersection after selecting the wrong turn lane.

These examples ride on the fact that other human drivers will be aware of your mistake and that they will react to it, and not behave like autonomous drones with preprogrammed paths. They give you some leeway for it and not a middle finger, and even move aside themselves, or they will react but will yield anyways like for the second example.

On Ideal Software Development Workflow

2020-08-19T00:00:00+00:00

Is it possible to harness unicorns? But ideally.

“The Best Part Is No Part”

What got me thinking on this topic was the remark think repeated by Elon Musk on several occasions, but most notably during Starship presentation last year that goes like -

The best part is no part. The best process is no process. It weighs nothing. Costs nothing. Can’t go wrong.

Basically, the way I see it is that everything that is shown as unnecessary will encumber either the development process or the product, and the product in the subsequent use.

What is of interest here is the hidden cost in the development process itself. As an end user, you can point at the product and state that this or that may not be needed, but there’s that background effort invested in realizing the product, which may have had its own “extra” unused parts, removal of which could lead to faster development, better devised product or just less burn out.

Ideal Software Development Workflow 🦄

The question that comes here is, how to reach this ideal development process? The common approach is minimization of most notable waste, eg. what Minifesto proclaims. Other approach would be to go from the idealistic development workflow, however unlike may it seemed, then identify possible violations of it in the context of current project and then select strategies or the workflow enhancements to mitigate these violations.

How would that look like? Ideal workflow eg. would be to:

Collect requirements in full, without any ambiguities, conflicts or omissions.
Based on these requirements and preexisting knowledge, a complete design of a system is devised, and pre-existing tools/libraries/frameworks are selected so that they enhance development and remove work duplication to the full.
In line with the product design, a support infrastructure for development is identified, and a development infrastructure is set up so that there is no need for further enhancements or changes.
Based on the given design, necessary missing software components are created.
Newly created code is seamlessly and without conflicts integrated with pre-existing software components.
Resulting software product is then integrated in the operation environment without conflicts and issues.
Work done. Profit.

What may seem is that the given outline looks too naive. And, for the majority of the projects it is. But, the core assumption holds, that if there are no violations to it, it would work. This is also relevant for the steps perceived as omitted, like maintenance. If the product is ideal, it would not require maintenance.

Dealing with Complexity

Onto the violations. What happens here is that due to the unaccounted factors it is possible that the given ideal workflow will not satisfy the project development needs. It may be due to the changing or incomplete requirements, understanding of the system, lack of preexisting knowledge in domain, or regarding the suitable tools/frameworks/libraries. It may be due to the distributed nature of development, where the synchronization issues between team members appear.

Whatever the reason, the idea is to see these factors as something that the ideal development workflow is not prepared to address, and mitigation strategies should be added to it in order to do so. Eg. in case of a possibility of having incomplete requirements, mitigation mechanisms could be creating the problem domain model and testing the requirements on it, or creating a mock-up product for asserting the requirements. In case of possible integration issues, interface specification and incremental development may be the solution.

The main idea behind this is to make all ideal development workflow violations visible, and to state mitigation strategies explicitly. This way ideal workflow in combination with mitigation strategies becomes a verifiable minimal development workflow based on the current understanding of the project.

As a side benefit, purpose of development workflow tools/processes becomes apparent with this, and they will not be seen as an externally imposed hurdle. Also, this allows for building a certain “repository”, or toolkit of useful tools to be applied in particular development scenarios.

What About Real-World Application?

The final question is the applicability of this thought out approach? Even Musk may say “The best process is no process”, but the feasibility of keeping true to that word is what matters. Guess that the best way to achieve it is:

To have an extensive knowledge of tools at disposal (toolkit).
Do not over-anticipate the issues, but develop the mitigation on fly.

Further than that, useful real-world experience examples would be beneficial.

Line Length Limit

2020-07-29T00:00:00+00:00

1   5    10   15   20   25   30   35   40   45   50   55   60   65   70   75
80   85   90   95   100  105  110  115  120
|...|....|....|....|....|....|....|....|....|....|....|....|....|....|....|....|....|....|....|....|....|....|....|....|

Table of Contents
Version History
Overview
A Look at The Problem
Making a CPL Limit Decision
1. Factors
2. Decision Examples
Glossary
CPL Limit Examples
Resources

Version History

Date	Description
2020-07-29	Initial version

Overview

Some time ago while looking through the Internets, a Reddit post on Linus’ rant about 80CPL limit caught my attention. The gist was that 80CPL is a thing of a past, and “more reasonable” limit should become a norm, like 120CPL or so.

This got me thinking on, what actually constitutes a justification for “more reasonable” limit, or, should limit be imposed at all?

This analysis tries to answer -

If and when limiting line length for text files matters
Also in that context - What would be the appropriate line length limit?

First part of this post is dedicated to the perceived origins of the problem, and to describing the underlying problem domain. It is followed next by a summary of relevant factors when choosing if and what CPL limit to impose.

Also, at the end there is some other useful information, like glossary used for terms here (like CPL), some CPL limit examples, and a resource list.

A Look at The Problem

Trigger for delving into the topic are the statements in discussion groups/blog posts like -

Pro banishing 80CPL:
- 120CPL is the magic number.
- Why stick with 80CPL when we’re not using 80CPL screens? 4K/dual monitor setup is a norm today.
- 80CPL makes code unnecessary long.
- 80CPL encourages naming abbreviation which reduces readability.
- Line breaks limit use of line-based tools.
Pro sticking to 80CPL:
- Diff viewers use is easier with lower limit.
- 80CPL supports easy side-by-side editing (if you don’t want to accept word wrap).
- 80CPL supports easy font enlargement for presentation/video recording.

One important thing mentioning here is that the bulk of these examples relate to code files, seeming to stem from code file lines becoming too long, but CPL rules apply to document files, too. Still, these examples may be used as a starting point to developing the view on the problem domain. Based on them, following generalized use cases are derived -

Optimal CPL when writing code in a single-window IDE.
Optimal CPL when writing code and having to reference some other location in source tree, in a multi side-by-side viewport setup.
Not being able to wrap long lines when performing code diff in VS Code, GitHub, Bitbucket.
Issue with CPL limit - Grep expressions not working across multiple lines.
Reading/modifying text files over limited-width terminal (embedded system serial port, SSH).
Reading/modifying text in an atypical work environment (eg. on a mobile phone via Markor).
Reading preformatted text in a browser (eg. source files or discussion group posts).

Text File Element Types

What is of interest is that the treatment of text files should not depend on the text file type per se, but it’s on the elements it consists of. These elements determine the way text is read, the approach on how it should be laid out, how it is accessed.

Here, the identified text file elements are listed -

File type	Element	Treatment
Code	Source code	Code
Code	Comment	Breakable text
Document	Paragraph	Breakable text
Document	Code block	Code
Document	Code block comment	Breakable text
Document	Table	Non-breakable text
Document	Hyperlink	Non-breakable text

Thing worth recapping here is that for a single text file, there may be different rules, depending on the text file element, eg. document table may have CPL limit but a document paragraph may have no CPL limit.

Text Operations

Next what impacts the treatment is the way text is used. In this regard, the following text operations are identified, as well as the tools that may be used -

Operation	Example tools
Navigation	Vim, Notepad
Inserting/modifying	Notepad, VS Code, Vim through terminal
Reading	Terminal `less`, browser render, VS Code
Comparison (diffing)	VS Code, GitHub/Bitbucket, terminal `diff`
Searching	Terminal `grep`

Most appropriate text file element CPL limit depends on the operation, but also on the tool employed, as well as the environment in which the tool is used. Eg. diffing with diff produces valid result regardless of CPL limit, where other tools do not have support for diff wrap and require horizontal scrolling, except if they do not use a sufficiently wide viewport.

This lead further to having a development environment determining, in combination with CPL limits, feasibility of the given operations. Development environment would consist out of -

Display(s) size (and orientation together with this).
Viewport size
Text editor:
1. Inherent operation model.
2. Configuration.
3. Reserved viewport space (eg. sidebar, scrollbars, minimap, line numbers, diff prefix).

Contributing Factors

In addition to the operation and text operation environment, there are some further side factors that may impact the usefulness of imposed CPL limit rules.

User base size - Single user vs. multiple users (with potentially disjunctive development environments).
Tabs vs. spaces and tab indentation value.

Making a CPL Limit Decision

Factors

Here’s a piled up list of observed factors that may contribute to a certain CPL limit decision.

Enforcing code writing style through CPL limit:
1. More concise variable names.
2. Local aliasing.
3. Condition nesting.
4. Long expressions rework (no sausage expressions).
Cost of adhering to a CPL limit when creating/modifying text:
1. Having to break long statements when writing code.
2. Having to make document tables fit.
3. Switching to an atypical work environment which is not configured/tailored to the imposed CPL limit (eg. editing files on a mobile phone or Web UI).
Readability:
1. Certain document line length range is optimal for reading convenience.
2. Same holds to a certain degree for code.
3. Wrapping of long lines depend on the text editor, and may produce unreadable output, eg. document tables or code.
4. If wrapping is not applied, then user need to perform horizontal scrolling to read the text which breaks the flow. This is esp. relevant to diffing. There:
  1. VS Code, Gitk, GitHub, Bitbucket, TFS do not wrap diff.
  2. git diff wraps diff.
5. For CPL limited text, if it is modified it may require rewrapping which will introduce modifications to the otherwise unmodified text. This may make diffing difficult.
  1. On the other hand, this is also a case with a long non-wrapped line.
6. This depends on the indentation style and amount.
Flexibility:
1. CPL limit makes text rigid, and not able to adapt to the environment at hand (eg. smaller mobile viewport), or developer’s preferences.
Output reproducibility:
1. Specific line length, use of spaces instead of tabs, and no wrap results in reproducible output. This ensures that everyone has the same representation on the text file.
2. Long line wrapping depends on the viewport configuration, and may change as the viewport is resized leading to different text representation even for the same user.
Ambiguity:
1. There’s a certain expectation that each text line fits into one displayed line. When this is broken due to line wrapping, there’s a risk that user will interpret wrapped lines as new text lines. This may result in the confusion when reading code statements, or the interpretation how the markup document file will be rendered.
Stimulating certain work environment:
1. Lower CPL limit stimulates use of narrower viewport, which leaves horizontal space for other windows, or stimulates multiple side-by-side viewport use.
Usability:
1. If document file paragraphs are not broken, this makes grep search for more than a single word queries possible.
2. Large CPL limit may prove diffing difficult as the tools usually don’t wrap text, so horizontal scrolling is needed.
3. Text wrapping may impede with normal workflow for some tools (eg. Vim jk navigation).
Development environment setup:
1. No CPL limit enforcement eases up setup.
2. Some environments even cannot be configured for the given CPL limit (eg. mobile edit environment).

Decision Examples

[TODO]

Glossary

Term	Meaning
Text file	Non-binary file containing readable text
Code file	Text file containing source code
Document file	Text file containing plaintext or markup text
CPL rule	Requirements on text file line length
CPL limit	Allowed number of characters per line
Text editor	Tool for editing or viewing a text file
Viewport	Screen area displaying text file content
Word wrap	Text editor ability to break long line to fit viewport

CPL Limit Examples

CPL	Note
70	Oracle’s Java Coding Convention - Documentation code examples
72	Python PEP 8 - Doctrings or comments
79	Python PEP 8 - Maximum
80	Punch cards and teletypes
120	Widely accepted better-than-80CPL limit

Resources

reddit_linus : Discussion on Linus’ post on line length limit.
baymard_line-length : Article on optimal length of text line.
nick_80cpl : Benefits of 80CPL with large monitors.
stack_optimal-width : (Lack of) info on studies on optimal code width.
python_pep8 : Python Coding style guide - line length.
stack_80cpl : Origin of 80CPL limit.
oracle_java-code-conv : Oracle’s Java Coding Conventions - Check 4.1.

A Take on GCC Linux Cross-Compilation Compatibility

2020-05-25T00:00:00+00:00

arm-linux-gnueabihf, almost usable for all ARM platforms.

Table of Contents
Version History
Overview
Cross-Compilation 101
Cross-Compilation Parameters and Compatibility
1. Parameter Violation Effects
Linux Shared Libraries
GCC
Potential Use Cases
Resources

Version History

Date	Description
2020-05-27	Initial bare-bones post

Overview

This topic started as a search for a more modern GCC version for cross-compiling the programs for Raspbian, and then turned into an all-out compatibility research on what it takes to cross-compile software, and be sure it’s going to run without issues on the target platform. The original topic got its own dedicated guide, and the rest of stuff collected on cross-compilation compatibility got slated for a dedicated analysis post, that is, this one.

This is a look on what consists a cross-compiler toolkit for a Linux target, what are its specifics, and as the subtitle says, why is a cross-compiler tied to a particular target even if the toolkit name prefix sounds generic enough so that it can cover a multitude of platforms. And at last, why that sometimes may be the case.

As a process to answer that question, here’s the outline of covered topics -

First there’s a refresher on cross-compilation
There’s a look on what are the relevant parameters for cross-compilation
A look at what makes a shared libraries compatible
What is the GCC cross-compiler build process
What are the assumed pitfalls of cross-compilation toolkit use

Cross-Compilation 101

Cross compilation in a nutshell is a process of building code on one machine that is to be executed on another (potentially incompatible) machine.

As this process considers using different machines (execution platforms, that is), here’s a recap of these -

Machine	Purpose	Built artifact
Build	Builds the compiler	Cross-compiler toolkit
Host	Runs the compiler	Cross-compiled user program
Target	Executes the compiled code	-

And here’s a recap of different build types and the corresponding machines -

Build name/Machine	Build	Host	Target	Example
Native	A	A	A	Local compilation
Cross	A	A	B	For MCU target
Cross-native	A	B	B	-
Canadian cross	A	B	C	-

Cross-Compilation Parameters and Compatibility

Here’s a summary of parameters relevant for creating a cross-compiler toolkit. All parameters are related to a target machine.

Parameter	Example	Compatibility note
CPU architecture	ARM	Incompatible
CPU architecture variant	ARMv6	Backward(1)
ABI	EABI	Variants inside ABI(2)
Object file format	ELF	Depends on format, linker
OS API	Linux API	Incompatible
OS API version	Linux API 3.2.0	Backward for Linux
Standard C library	glibc, uclibc	Incompatible
Standard C library version	glibc 2.28	Backward for glibc
Compiler runtime library	libgcc	Incompatible
Compiler RT library version	libgcc 7.0.0	Backward for libgcc

Notes:

Eg. for ARM AArch32, or x86 sans extensions.
Eg. EABI hard-float and soft-float variants are compatible between themselves.

Parameter Violation Effects

Now, with knowledge what parameters are all involved when making a cross-compiler, let’s see what are the potential effects of setting an incorrect parameter value

Parameter violation	Effect
CPU architecture	Code execution will crash
CPU architecture variant	Crash on unsupported execution
ABI	Crash on linked library call
ABI	Crash on an OS system call
Object file format	Linker/loader won’t accept the file
OS API	Any system call will fail
OS API version	Unsupported system calls will fail
Standard C library	Linking will fail
Standard C library version	Linking will fail
Compiler runtime library	Linking will fail
Compiler RT library version	Linking will fail

Linux Shared Libraries

Shared libraries in Linux go under three names -

Name	Naming example	Note
soname	`libstdc++.so.6`(1)	Symlink to real name
Real name	`libstdc++.so.6.0.28`	-
Linker name	`libstdc++.so`	Symlink to latest soname

Notes:

Standard C libraries don’t start with lib.

As for the shared library and their symbols versioning, here are the notes -

Linker uses library represented with latest library (one represented by linker name).
Version number specifies interface, and it’s possible to have multiple sonames for different major library versions.

API inside library is also versioned, with newer libraries adding API revisions, while keeping the old API. Example from libc-2.31.so -

00149b80  7039 FUNC    GLOBAL DEFAULT   16 glob64@GLIBC_2.2
0014b900  7039 FUNC    GLOBAL DEFAULT   16 glob64@GLIBC_2.1
000d1620  7039 FUNC    GLOBAL DEFAULT   16 glob64@@GLIBC_2.27

GCC

Cross-Compilation Toolkit Components

Here’s an overview of components, and how and where they’re gonna be used -

Component	Use	Use location
Binaries	Perform compilation, linking	Host
C static libraries	Link with compiled program	Host
C dynamic libraries	Link with compiled program	Target
C libraries headers	Program compilation	Host
Dynamic linker/loader	Load and link with dynamic libs	Target
GCC static libraries	Link with compiled program	Host
GCC dynamic libraries	Link with compiled program	Target
CRT object files	Program compilation	Host
GCC headers	Program compilation	Host
Linker scripts	Produce object file	Host
Documentation	Get to perform tasks	Host

GCC Toolkit Build Process

Build by steps -

Tool	Note
Binutils	Linker and assembler most importantly
GCC dependency libs	Needed for build-time calculations
1st stage GCC	Static C only, no libc
Libc	Build both dynamic linker/loader, and C library
2nd stage GCC	Libgcc built here; Need to know libc type/ver(1)

Notes:

Changing Libc requires GCC rebuild!

Most notably, used architecture, ABI is used for building libc and libgcc. This means that:

Whatever architecture configuration modifications are performed during application build, originally built libc and libgcc are still going to be used.
Any change needed necessitates rebuild for both libc and libgcc.

Cross-Compilation Toolkit Install Directory Contents

GCC requires binutils names without prefix
GCC cross-compiler naming prefix - Autoconf system canonical name - <arch>-<vendor>-<os>-<libc/abi>.
Standard C++ library considered as a build-up to C library
GCC library headers
Linker scripts

Potential Use Cases

Build GCC using prebuilt Glibc - Won’t work
Build newer GCC than the one used on target - Works, although libgcc compatibility not guaranteed.
Build with newer Glibc than the one used on target - Works, execution risky.
Using toolchain built for different machine - Works, execution not guaranteed.

Resources

bootlin_cctc_slides : “Anatomy of Cross-Compilation Toolchains” presentation slides.
bootlin_cctc_rec : “Anatomy of Cross-Compilation Toolchains” presentation recording.
gcc_mach-dep-opt : GCC machine-dependent options - Machine architecture, ABI, etc.
gcc_build-config : GCC build - Configuration options.
bootlin_emblin_slides : General information on cross-compilers.
tldp_so : Naming, version info on shared libraries.
redhat_lib-if-vers : How library does interface versioning.
autoconf_gcc-prefix : Cross-compiler GCC prefix format.
gcc_releases : GCC release history.

Cross-compile Program for Raspbian

2020-05-23T00:00:00+00:00

“Hello, Raspbian from a more capable build machine!\n”

Table of Contents
Overview
Cross-Compilation Toolchain Selection
Environment/Project Setup Procedure
Building The Project
Resources
To-Do

Overview

Goal is to set up a cross-compilation environment on an Ubuntu-based machine and build a “Hello, world!” CMake project for the target.

Setup used for testing is -

Component	Value
Build OS	Ubuntu 20.04 LTS
Target OS	Raspbian Buster Lite, February 2020
Target platform	Raspberry Pi 3 Model B

Cross-Compilation Toolchain Selection

Here, the main point is to obtain the cross-compilation toolchain for RPi. The important thing is that this toolchain produces binaries which are compatible with the execution environment. This is a topic to be covered, but for now following toolchains will work:

Source	GCC version	Source
toolchain_rpi	4.9.3	Raspberry Pi Foundation
toolchain_pro	8.3.0	Alternative build by Pro

Notes:

RPi repo has several toolchains. Select one per this article.

Environment/Project Setup Procedure

Checkout this repo with test CMake projects - GitHub link.
Set up the RPi test project per its readme.

Building The Project

Run configure/build for CMake (<project>/build$ cmake .. && make).
Built binaries are in CMake subproject subdirectories.

We can verify they have been cross compiled by running file -

<project>/build/hello-raspbian-cross-pi-gcc$ file HelloRaspbianCrossPiGcc 
HelloRaspbianCrossPiGcc: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 3.2.0, with debug_info, not stripped

Resources

toolchain_rpi : Overview of MCU and the ESP32-based boards.
toolchain_pro : Page with pretty much everything on ESP32.
hello-vscode-cmake : Repo with test CMake project.
pro-toolchain-build : Article on building a GCC toolchain for RPi.
cmake-cross-compile : How to setup CMake project for cross compilation.
deardevices_rpi-cross-compile : How to setup RPi provided toolchains.

To-Do

Checklist for toolchain suitability for the given target.

Git Guide

2020-05-15T00:00:00+00:00

Some tips and tricks on Git collected here and there.

Table of Contents
Credentials Management
Line Endings Configuration

Credentials Management

The question addressed here is - What I need to do to get my credentials stored in a safe way, and avoid getting prodded for them every time I access a remote like GitHub. And, switching to SSH key pair authentication is either inconvenient or not possible.

The procedure is this:

Set up low-risk credentials - This is not Git-specific topic, but useful nevertheless. The idea is to use access tokens instead of an account password when accessing the site repo (GitHub, TFS). These tokens may have limited permitted operations scope, thus lowering damage done by them being exposed. Process here is:
1. Find a way to generate a token on the used remote - Eg. for GitHub that can be done over https://github.com/settings/tokens.
2. Use generated token instead of the account password when accessing that site repos.

See what credential helpers are available for Git installation -

$ git help -a | grep credential-

Some common options here are:

Helper	Description
`cache`	In-memory temporary storage
`store`	Plaintext file (usually `~/.git-credentials`)
`manager`	Windows Credential Store
`wincred`	Deprecated in favor of `manager`

Set the credential helper -

$ git config --global credential.helper <helper>

Line Endings Configuration

Ok, this is a common topic but still worth reiterating. Basically, aim is to get a consistent line endings for text files, both in the repository and in the working directory. These would be:

All LF in the repo.
On Windows machine, all CRLF except for the files marked as LF (eg. Bash scripts).
On Linux machine, all LF except for the files marked as CRLF (eg. Visual Studio project file).

Steps to do this are:

Set line ending conversion for commit/checkout:
1. (on Windows) Set core.autocrlf, so that all checked out text files have CRLF newline -
```
$ git config --global core.autocrlf true
```
2. (on other LF-using platforms) These platforms do not require conversion from Git default LF line endings, so:
  1. See that core.autocrlf is unset or false as this option is intended for Windows use -
    $ git config --get core.autocrlf
  2. In addition, check that core.eol is unset or native as we will be using native line endings -
    $ git config --get core.eol
Set repo configuration for files that should keep their line endings regardless of the platform (eg. Bash scripts when checked out on Windows, or Visual Studio project when checked out on Linux):
1. Create .gitattributes file in the root directory for files to be configured.
2. State expected line endings for required paths, eg. *.sln text eol=crlf.

Renormalize line endings, and commit renormalized endings (if there is any) -

(repo)$ git add --renormalize .
(repo)$ git commit -m "Renormalize line endings"

Visual Studio Code and CMake (and Other Tools)

2020-05-14T00:00:00+00:00

Visual Studio Code and C/C++ project development? Let’s see how they work together.

Separation of Paths

When started with C/C++ software development, I didn’t give much thought on the integration of components that make an IDE. Basically, what I did is to fire it up, create a project which automagically set up build configurations, altogether with toolchain, sources and includes configuration, and then - code away! Code assistance operations like disabling inactive code lines, code completion, jump to definition worked out of the box, as well as highlighting of syntax and compile errors.

This mode of operation translated seamlessly to the MCU embedded development, with every vendor providing a complete tool ecosystem for creating a firmware to be executed on their hardware platform portfolio. But, on the other hand, Linux application development diverged in the other area, one with clear demarcation between the code editor, and the build system. There, former was seen as independent from the latter, and furthermore, oblivious to the configuration provided by it. And, even with the editor providing facilities as aforementioned code assistance and highlighting, it was often a manual process to keep its perception of software project configuration in line with the build system’s one. This led to being wary of these “assistance” tools as they could provide a false image on the project code, and went as far as ditching them altogether and demoting the code editor to a intelligent text editor at best, dumb notepad at worst.

Enter VS Code

There were previous interests for regaining this connection between a development environment and a code editor, but with short longevity. This state of affairs went on as I got introduced to a VS Code. I’ve ditched it first as a coding hipsters kind of a tool, especially seeing it in light of web development where it first took ahold. Instead, I’ve stuck rather to a proven Vim text editing environment. But, gradually I transitioned to VS Code, mostly driven by liking of its workspace management and a suitable enough Vim extension. With time it became my main file projects management tool, providing almost all-in-one environment for needed operations.

So, with that in mind I’ve revisited the old topic of bridging a gap between a code editor and a the build system. The question was, can Visual Studio Code be connected with a C/C++ set up console-driven build system, and assist in the code development while relying on the project configuration specified outside of it?

Setting The Stage

First topic to clarify was, what would be the assumed build system to attach to? CMake here came out as a first candidate, as it is a popular-enough project configure/build/test/package toolkit, and with which I had most previous experience. As for the other options, plain Makefiles are also a good option but more suitable for simpler task specification, and other toolkits I haven’t considered at all.

Second topic was, what would be the expectations from the integration? From the top of the head first would be code assistance aligned with selected build configuration, that is with selected source files to be built, included includes and libraries, and project defines. Next in line would be interface for running build system tasks, then configuration of the debugger so that a debug session can be ran from VS Code. As a last addition, integration with code writing assistance tools, namely a formatter and a static analysis would be great, with latter again aligned to the current build configuration.

For formatting and static analysis choice fell on the Clang toolkit, ClangFormat and Clang-Tidy, mostly because they’re good free tools, with configuration specifiable in config files, and with good preexisting integration to VS Code and CMake.

Add Extensions to The Mix

Basically, to get things running all I needed were two extensions for VS Code - ms-vscode.cpptools for code assistance and formatting, and ms-vscode.cmake-tools for CMake integration. With them, and with other prerequisite packages installed in the evaluation VM (gdb, clang-format, clang-tidy), I was set to go.

Initial Impressions

In short, positive. Code assistance works for the opened projects, even if CMake contains subprojects with same-named symbols. Interface also works without a hitch - Configuring project, building it, running particular subproject. Debugging, too, non-compiled code gets ignored just fine.

For the additional tools, VS Code invokes formatting per given configuration, while static analysis is enabled through CMake and ran before the compilation unit build. What made me smile is that the reported analysis findings are shown in the code editor window altogether with squiggly lines.

What Next?

Well, good initial findings are there, but there’s still that nagging feeling about disjuncture between the build system and code editor. Guess it will need some additional test projects and the use of this integration on the actual projects to dispel that impression.

If you want to check it out, current VS Code workspace with test projects I have hosted on GitHub - here’s a repo link.

Heltec ESP32 Door Lock Notifier

2019-08-11T00:00:00+00:00

Did I turn on the notification that my apartment door is locked?!

Table of Contents
Overview
ESP32
1. Heltec WiFi LoRa 32
2. (Arduino) IDE
Door Lock Notifier
Resources

Overview

About a year ago I’ve bought 2 Heltec’s ESP32 modules, and after checking them out if they work put them in a drawer where they stayed ever since. Thought it would be good to dust them off and give them a try, especially that I haven’t worked with Espressif microcontrollers before.

After some initial research on the MCU/module, an idea came to me to make as a mini learning project a door lock notifier. Ever once in a while I have that “did I lock the door or not?” moment with the apartment door, so that would be a good place to put the these modules to use.

ESP32

First, onto ESP32. In short, this is an MCU with 32b CPU and a whole bunch of peripherals, most notably integrated WiFi/BT module. Quick overview can be found on a Wiki page.

Its predecessor was ESP8266, most notably known through NodeMcu. For a comparison what has changed, there’s a comparison with ESP8266.

As for the ESP32-relevant information, the best repository is the esp32.net. There you can find -

Chip/module portfolio
How to do the development - Arduino IDE, ESP-IDF

In addition to this, TRM can be found on the Espressif official webpage.

Heltec WiFi LoRa 32

Module that I have at hands is the Heltec’s WiFi LoRa 32 module. Here’s how it looks live -

In addition to all the ESP32 goodies, it also incorporates the following features, first and the last found to be useful for this project -

A small OLED display
SX1276/1278 LoRa chip
Lithium battery interface

Useful stuff on the module (pinout, schematics, lib examples) can be found on Heltec repos -

(Arduino) IDE

Next, the question is regarding the IDE for ESP32 modules, which one can be used. There are two reasonable options -

Using Heltec’s ESP-IDF
Using Arduino IDE

I’ve went with the second option because it seemed as a faster way to perform some quick prototyping. Here are some resources on setting up the work environment -

Door Lock Notifier

As for the mini project, the concept is following:

Let the module somehow detect that the door has locked/unlocked.
Upon detecting the change, connect to local Internet-enabled WiFi AP, through it to the IFTTT server, and send a POST request containing the update message.
IFTTT will then using its Telegram bot generate a new message in the messenger.
This message can then be observed via phone.

Here’s an overview of the system components -

Development Decisions

Detect Door Lock/Unlock

First question that appeared was - How to detect door lock/unlock event?

The preferred way would be to use the movement of the lock deadbolt to determine the transition between locked/unlocked state. This has a benefit that the perceived state can quite certainly correspond to the actual state.

As for detecting this transition, the straightforward way would be to use a micro switch installed in the door frame, so that it gets closed by a deadbolt going in the frame. But, the cheaper and faster option for the prototyping purpose is to use the fact that deadbolts are made of metal in order to be strong enough. Metal deadbolt, being conductive, can itself serve as a part of a switch. Additional hardware would then require just two additional leaf contacts leading to the used Heltec board pins. Here’s how it looks like from side -

Connection to IFTTT Server

For connecting to the IFTTT server WiFi Client example sketch has been used, spiced up with the example on how to do HTTPS connection.

Module Autonomy

As the module is running on a battery, it would be preferable to lower its power consumption so that it can operate for the extended period of time. Of course, connecting it on an USB adapter would be the best solution, but there were none close to the door and stretching the cables over the wall wasn’t an option.

Fortunately, ESP32 supports several power models, one of them being a deep-sleep one, with -

~6.5uA current consumption
RTC memory persistence
External wake-up source
And most importantly for fast prototyping, Arduino example sketch

So, the solution was to use the door lock detection switch as a wake-up source, and set the wake-up level opposite of the last read out. Deep sleep Arduino example has been used as a basis for it.

System Components

Notification Mechanism

Notification chain for IFTTT looks like -

Set up web request with esp32_reading event name.

Send message to Telegram private chat with IFTTT bot with following format:

What: {{EventName}}<br>
When: {{OccurredAt}}<br>
Door status: {{Value1}}<br>

Notifier

Hardware outline is this -

Connect Heltec board to battery and leaf-contact simulating wires.
Use breadboard to hold everything together.

Here’s how it looks like -

Here is the connection table for the used pins based on the pinout diagram:

Pin	Use
GND	Leaf wire 1
0 (PRG button)	Leaf wire 2

As for the Arduino sketch, it’s uploaded to the GitHub repo.

Results

Here’s the live demonstration - YouTube video.

During the use, two main observed issues were:

Sporadic triggering.
Battery drain.

Resources

esp32-wiki : Overview of MCU and the ESP32-based boards.
esp32-net : Page with pretty much everything on ESP32.
node-mcu : NodeMcu page.
esp-comparison-table : ESP8266/ESP32 comparison table.
esp32-hardware : Comprehensive overview of ESP32 chips and modules.
esp32-development : Comprehensive overview of ESP32 development tools.
esp32-trm : ESP32 technical reference manual.
heltec-wifi-lora-32 : Heltec WiFi LoRa 32 module info page.
module-resources : Module schematic, pinout.
esp32-arduino-install : Arduino setup for ESP32 boards.
heltec-esp32-arduino-lib : Installation procedure for Heltec ESP32 Arduino library.
arduino-vscode-extension : VS Code Arduino extension.
wifi-sketch : ESP32 WiFi client sample sketch.
https-get-request : Guide how to create GET request over HTTPS.
deep-sleep-sketch : ESP32 deep sleep sample sketch.
sketch-repo : Sketch repo.

Kibihrchak’s Best Webpage

VS Code Remote Development

Initial Encounter with VS Code Remote Development

Common Development Interface

Mobile Phone Cameras

Leveraging Computational Photography

Camera as a Code

Computational Photography is Cheating

Where Do We Go Next?

Autonomous Driving Observations

Driving Instructor to an Autonomous Driving System

“It is Human to Make Mistakes”

On Ideal Software Development Workflow

“The Best Part Is No Part”

Ideal Software Development Workflow 🦄

Dealing with Complexity

What About Real-World Application?

Line Length Limit

Table of Contents

Version History

Overview

A Look at The Problem

Text File Element Types

Text Operations

Contributing Factors

Making a CPL Limit Decision

Factors

Decision Examples

Glossary

CPL Limit Examples

Resources

A Take on GCC Linux Cross-Compilation Compatibility

Table of Contents

Version History

Overview

Cross-Compilation 101

Cross-Compilation Parameters and Compatibility

Parameter Violation Effects

Linux Shared Libraries

GCC

Cross-Compilation Toolkit Components

GCC Toolkit Build Process

Cross-Compilation Toolkit Install Directory Contents

Potential Use Cases

Resources

Cross-compile Program for Raspbian

Table of Contents

Overview

Cross-Compilation Toolchain Selection

Environment/Project Setup Procedure

Building The Project

Resources

To-Do

Git Guide

Table of Contents

Credentials Management

Line Endings Configuration

Visual Studio Code and CMake (and Other Tools)

Separation of Paths

Enter VS Code

Setting The Stage

Add Extensions to The Mix

Initial Impressions

What Next?

Heltec ESP32 Door Lock Notifier

Table of Contents

Overview

ESP32

Heltec WiFi LoRa 32

(Arduino) IDE

Door Lock Notifier

Development Decisions

Detect Door Lock/Unlock

Connection to IFTTT Server

Module Autonomy

System Components

Notification Mechanism

Notifier

Results

Resources