# 探码科技-发开日志

## 我们用什么操作系统开发?

[MacOS](https://www.apple.com/macos)、[Ubuntu](https://www.ubuntu.com/)

## 我们用什么IDE编写代码?

[VS Code](https://code.visualstudio.com/)

配置文件：<https://gist.github.com/xiaohui-zhangxh/6ef523f273438ebb227337a4dd4efdcd>


# 代码规范

对于编程的代码规范，我们都遵循国际习惯，做到规范统一，每个程序员都能看懂他人的代码。

* [HTML & CSS](https://google.github.io/styleguide/htmlcssguide.html)
* [Javascript](https://github.com/airbnb/javascript)
* [Ruby](https://github.com/JuanitoFatas/ruby-style-guide/blob/master/README-zhCN.md)
  * 使用Ruby 2.3+
  * 使用`dig`方法深度获取Hash的值
    * `params[:search][:keywords]` 不推荐
    * `params.dig(:search, :keywords)` 推荐
* [Ruby On Rails](https://github.com/JuanitoFatas/rails-style-guide/blob/master/README-zhCN.md)
  * Assets资源引用途径的优先级：[RubyGems](https://rubygems.org/gems/bootstrap) > [RailsAssets](https://rails-assets.org/#/components/bootstrap) > [NPM](https://www.npmjs.com/package/bootstrap) > Download


# 前端


# Bootstrap 模板


# 图表类

{% embed url="<http://coderthemes.com/flacto/light_red_1_dark/index.html>" %}

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LAHg8_1eiP1zOaqkqQT%2F-LAHhJGaU8x8UMIKzvDB%2Fimage.png?alt=media\&token=542a9b15-6af8-4de4-93c3-a87a3ec36b4f)


# Profile 类页面

{% embed url="<https://wrappixel.com/demos/admin-templates/material-pro/horizontal/app-contact-detail.html>" %}

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LAHhTpdNsElLYqqey76%2F-LAHpS-6XRmitCgpElQJ%2Fimage.png?alt=media\&token=e1f3bfff-0393-4ab4-bbff-e37aa7fbddae)


# JS Chart图表

## morris.js

> 特点：调用非常简单，没有过多的订制

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LAIGG_iuBN7cFNVUeUx%2F-LAIGmg1LL_g1hcHyXDo%2Fimage.png?alt=media\&token=fdc7f8f7-a09e-45d5-ac79-3ba80bd52351)

## D3.js

{% embed url="<https://d3js.org/>" %}

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LJRS4cc3fv95TQ2gBgz%2F-LJRX1ifNmxjdFWkAEZy%2Fimage.png?alt=media\&token=126a0430-1477-4ae0-a575-077e487c70d9)

## C3.js

{% embed url="<https://c3js.org/>" %}

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LJRS4cc3fv95TQ2gBgz%2F-LJRXHgHduYPX0VTIDTz%2Fimage.png?alt=media\&token=4c0b2866-6a9a-4f6a-ba81-8d4e3b6b24ef)

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LJRS4cc3fv95TQ2gBgz%2F-LJRXMpkb_e-hQqwN9TJ%2Fimage.png?alt=media\&token=1b4a799a-af30-4f03-acaa-cb7c693e4148)


# 图片库

### 东方IC

图片种类全 <http://creative.dfic.cn/>

### 500PX

出色的摄影图片 <https://500px.com/>

### Pixels

免费高品质图片，尺寸全 <https://www.pexels.com/>

### Pixabay

正版免费图片视频库 <https://pixabay.com/>

### LogoSc

Logo制作 <http://www.logosc.cn/>

### TinyPng

免费图片在线压缩 <https://tinypng.com/>

### MagicMockup集合

苹果设备场景模型图 <http://magicmockups.com/>

设备单一模型图 <https://dimmy.club/>

设备模型图 <https://mockuphone.com/#ios>

设备模型图 <https://smartmockups.com/>

在线Ps(ubuntu同学) <https://pixlr.com/express/>

在线海报/Banner制作 <https://crello.com/home/>

食物单品图 <https://www.foodiesfeed.com/>

强大的免抠图素材 <http://pngimg.com/>

&#x20;3D模型制作 <http://threed.io/>

&#x20;模型图PSD <https://freebiesbug.com/psd-freebies/mockups/>


# Icon图标库

[ICONFINDER图标库](https://www.iconfinder.com/)

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LAXHlzNVb13jeD7Sjuz%2F-LAXI6eEOjn5m7WtjxsX%2Ficonfinder.png?alt=media\&token=45dcb3e4-adfb-4aa7-bba4-51f5b0897a6d)

[阿里巴巴ICONfont](http://www.iconfont.cn/)

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LAXJh7p_wYfHDeh2dFU%2F-LAXKE1KwBD0tq5SorQ2%2Ficon-font.png?alt=media\&token=81df6406-f1c0-4fdf-81bd-f8aa4c0481b9)

[FontAwesome图标库](https://fontawesome.com/icons?from=io)

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LAXKdUeiDps9I09nqOH%2F-LAXKs4XDZkU6K1_1ds8%2Ffontawsome.png?alt=media\&token=ad6385a3-0d0c-4c9d-bc3a-03d0fd72f7ff)

[FlatIcon图标库](https://www.flaticon.com/)

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LB1Oi6Sc_g4wYCXTNCS%2F-LB1Oy-bwcnwduL5zZKw%2Fimage.png?alt=media\&token=c72e1cbe-eb1d-49ef-b646-7d1766191cd5)


# Css3


# 字体+背景混合搭配

## 应用案例：

<http://www.sodacan.cn/>

<http://www.0900.cc/>

<http://highgradelab.com/stash/branding-agency/>

## 字体+背景混合搭配

最终效果图：

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LNVtU1_8q_O6em7rosv%2F-LNVupMkQ5pHDGHDbF5Y%2Fimage.png?alt=media\&token=1aa69165-0ca8-4d0a-b034-03058c2bbd1c)

HTML代码块：

```
<!-- html start -->
  <div class="tanmer__text__background animated fadeIn delay-2s" style="--image_url: url('./images/bg.jpg')">
    <h1 class="tanmer__text size__10">Hello World !</h1>
  </div>
<!-- end -->
```

CSS代码块：

```
<!-- animation css -->
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/animate.css/3.5.2/animate.min.css">

/* 可要可不要 */
html, body {
  height: 1000px;
  width: 100%;
} 

/* css start */
  .tanmer__text__background {
    width: 100%;
    height: 100%;
    background-color: white;
    background-size: cover;
    background-position: center center;
    background-attachment: fixed;

    position: relative;

    /* variable */
    background-image: var(--image_url)
  }

  .tanmer__text__background .tanmer__text {
    font-weight: bold;
    height: 100%;

    /* display flex */
    display: flex;
    display: -webkit-flex;
    justify-content: center;
    align-items: center;

    /* 颜色混合，黑色透明 */
    mix-blend-mode: screen;

    /* variable params2 初始值 */
    font-size: var(--font_size, 5vw);
    background-color: var(--background_color, white);
    color: var(--color, black);
  }
/* end */
```

### 扩展

最终效果图：

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LNYp6t2Dh5inZ1dN5Wh%2F-LNYp98-06S5mu2p8dzv%2Fimage.png?alt=media\&token=d2aec9fc-6dd0-402c-b07f-ec9d35069b0a)

HTML增加的代码块：

```
<!-- html start -->
  <div class="tanmer__text__background animated fadeIn delay-2s" style="--image_url: url('./images/bg.jpg')">
    <div class="tanmmer__position__mix__blend"></div>
    <h1 class="tanmer__text size__10">Hello World !</h1>
  </div>
<!-- end -->
```

CSS增加的代码块：

```
 /* tanmmer__position__mix__blend */
  .tanmer__text__background .tanmmer__position__mix__blend {
    position: absolute;
    width: 100%;
    height: 100%;

    /* variable params2 初始值 */
    mix-blend-mode: var(--mix_blend_mode, color-burn);
    background-color: var(--background_color, #03c9a9);
  }
```

#### 结构

* \<div  class="tanmer\_\_text\_\_background">
  * 描述：主要用于放置背景图片，使图片固定在页面中，不跟随滚动
  * 变量：在 style 属性中可以定义背景图片地址，例： --image\_url: url('图片路径')
    * \<div class="tanmmer\_\_position\_\_mix\_\_blend" >
      * 描述：层级关系 图片 <  当前 < 文字, 搭配 mix-blend-mode 渲染文字颜色。
      * 变量：包含--mix-blend-mode（见参考文档）、--background-color，在 style 属性中进行赋值
    * \<h1 class="tanmer\_\_text" >
      * 描述：处理文字展示效果
      * 变量：包含 --font-size、--background-color、--color，在 style 属性中进行赋值

> #### 参考文档：
>
> <https://www.youtube.com/watch?v=vs34f9FiHps&t=3783s> （mix-blend-mode youtube视屏）
>
> <https://developer.mozilla.org/en-US/docs/Web/CSS/mix-blend-mode> （mix-blend-mode MDN文档）


# tranform-origin + transform

自己可以先简单的实现 hover 默认的两种状态实现下滑线滑动

使用元素的 before 和 after , 定位到当前操作的为父级元素上, 效果前后利用 hover 和 transition 实现过度动画, 方便往下进阶。

### 达到的效果

采用 transition(过度效果)、transform: scaleX (缩放)、transform-origin(改变原点位置)，实现下滑线从左边近，右边出

#### 首先理解 hover 选择器的作用, 使用它时元素被分解成三个步骤

* hover 前(默认样式)
* hover 中(:hover 设置样式)
* hover 后(默认样式)

由此可以看出 hover 正常状态下只有两种状态

所以，必须要有一种方法，能够使得 hover 动画的进入与离开产生两种不一样的效果, 实现:

* hover 前设置原点位置 transform-orgin: right 0)、缩放大小 transform: scaleX(0), 隐藏 before 选择器
* hover 中(设置原点位置 transform-orgin: 0 0)、缩放大小 transform: scaleX(1), 显示 before 选择器
* hover 前(恢复 transform: scaleX(0)、transform-orgin: right 0), 向右隐藏 before 选择器

### 代码如下

```
<div class="box">Hover default</div>

.box {
  position: relative;
  left: 25%;
  width: 200px;
  height: 60px;
  line-height: 60px;
  color: #333;
  text-align: center;
  transition: color .5s;
  border-top: 2px solid #e2e2e2;
  border-bottom: 2px solid #e2e2e2;
}
.box::before {
  content: '';
  position: absolute;
  left: 0px;
  bottom: 0px;
  width: 200px;
  height: 2px;
  background-color: #000;
  transition: transform .5s;
  transform: scaleX(0);
  transform-origin: right 0;
}
.box::after {
  content: '';
  position: absolute;
  right: 0px;
  top: -2px;
  width: 200px;
  height: 2px;
  background-color: #000;
  transition: transform .5s;
  transform: scaleX(0);
  transform-origin: left 0;
}
.box:hover {
  color: red;
}
.box:hover::before {
  transform: scaleX(1);
  transform-origin: 0 0;
}
.box:hover::after {
 transform: scaleX(1);
 transform-origin: right 0;
}
```

### 参考文档

transform-origin 取值:

* x-axis 默认以元素框上边框为 x正半轴, 上边框与左边框的焦点为顶点; 水平偏移量有以下几种取值方式
  * left 等于 0 或 0%
  * center 等于 50%
  * right 等于 100%
* y-axis 以元素框左边框为 y正半轴, 左边框与上边框的焦点为顶点; 水平偏移量有以下几种取值方式
  * top 等于 0 或 0%
  * center 等于 50%
  * right 等于 100%
* z-axis 3D变形中会用到 详解见文档

transform-origin 实例解析:  <https://developer.mozilla.org/zh-CN/docs/Web/CSS/transform-origin>

transform-origin 坐标系图析:  <https://www.jianshu.com/p/fd44d21287ea>


# flex

阅读: <http://www.ruanyifeng.com/blog/2015/07/flex-grammar.html>


# 布局左边自适应，右边固定宽度

1. flex 左边自适应，右边固定宽度
2. 项目中踩坑集锦

> 简要介绍: 容器（flex container）和 项目（flex item）

## flex 左边自适应，右边固定宽度

```
// html
<div class="demo">
  <div class="left">
  </div>
  <div class="right">
  </div>
</div>

// style
.demo {
  border: 3px solid red;
  display: flex;
  align-items: stretch;

  // 左右空白比为 0:1，所以右边会分配到剩余空白的所有空间
  .left {
    background: #F7E8B4;

    // 宽度自动填充
    width: auto;

    // 有多余的项目空间，分配
    flex-grow: 1;
    height: 200px;
  }

  .right {
    background: #B4D3F7;

    // 设置宽度
    width: 200px;
    min-width: 200px;
    max-width: 200px;

    // 有多余的项目空间，不分配
    flex-grow: 0;
  }
}
```

效果图如下:

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LE8-JGl4GEY4ry04yps%2F-LE81JhZSLX3FtARqt_G%2Fimage.png?alt=media\&token=febbca3e-e257-49ac-9f80-3a87d7921a4b)

## 项目中踩坑集锦

接着以上的做法，应用到实际的项目需求中时，发生了下面👇意料之外的情况：

> 需求:  左侧包含的内容有标题、描述(超出并显示省略号)，右侧是一张固定宽高的图片。

根据需求建立如下图：

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LE8-JGl4GEY4ry04yps%2F-LE82QF7Vr8fCyjZ6ojl%2Fimage.png?alt=media\&token=010541dd-25a9-4b7b-8c09-3361447625d0)

\
现在就要解决的是描述(**超出并显示省略号) :**

```
.left {
  background: #F7E8B4;

  // 宽度自动填充
  width: auto;

  // 有多余的项目空间，分配
  flex-grow: 1;
  height: 200px;
  
  // 超出并显示省略号
  p {
    text-overflow: ellipsis;
    white-space: nowrap;
    overflow: hidden;
  }
}

```

如下图：

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LE82xKheI0pkr9WJepj%2F-LE83YS6kZbjF5vCV459%2Fimage.png?alt=media\&token=8966a46e-0fcf-4cb7-95bc-41fee5519fad)

可见左侧的宽度已经将右侧挤出可视宽度区域，这不是我们想要的。

**产生的原因：**

&#x20;    返回我们刚刚写的css，我们将左侧宽度设置为自适应 `width: auto;` p 标签样式中（超出并显示省略号）还有个宽度限制的样式没有设置，所以它会继承父级宽度，所以才会出现上图这种情况。

**解决方法：**

&#x20;   限制父级宽度并且父级宽度做到剩余空间自动分配。这句话搭配起来的意思就是left盒子 `width: 0; flex-grow: 1;`   将left盒子的宽度设置为 0 并将多余的空间全部分配给 left盒子, 这样 p标签的父级 就不会继承 left盒子样式中width宽度，继承的是 flex-grow 自动分配的宽度；这样需求中的问题也就解决了

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LE82xKheI0pkr9WJepj%2F-LE87b8YvA-pYhj5JXLA%2Fimage.png?alt=media\&token=cce050c6-254d-4ef9-a1a6-faaf3ab4831e)


# 用Sass颜色函数控制颜色

将要处理的颜色创建变量

```css
$base-color: #AD141E;
```

### 变黑 & 变淡 （Darken & Lighten）

原文：

> These two adjust the Lightness of the color’s HSL values. Sass will parse our hex color variable to hsl, then make the adjustments. You call them on the color with a percentage value between 0 and 100. I usually stay in the range of 3-20%.

翻译

> 这两个调整了颜色的HSL值的亮度。Sass将把我们的十六进制颜色变量解析为hsl，然后进行调整。你可以用0到100之间的百分数来称呼它们。通常停留在3-20%的范围内。

```
darken( $base-color, 10% )
lighten( $base-color, 10% )
```

![Darken & Lighten](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LB1QjQYDebF4yqHF6wh%2F-LB1SmiRcoL2Z1oeHPun%2Fdark_light.png?alt=media\&token=2c07798a-e875-40a5-a49f-9d959fddff99)

### 饱和 & 冲淡（Saturate & Desaturate）

原文

> These will will adjust the Saturation of the colors HSL values, much like Darken and Lighten adjusted the Lightness. Again, you need to give a percentage value to saturate and desaturate.&#x20;

翻译

> 这些将会调整HSL值颜色的饱和度，就像变暗一样，调节亮度。同样，你需要给出一个百分比值来饱和和饱和度。

```
saturate( $base-color, 20% )
desaturate( $base-color, 20% )
```

![Saturate & Desaturate](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LB1QjQYDebF4yqHF6wh%2F-LB1Ts4YbsT3PBkvQ3yQ%2Ftumblr_luv5ij6hvY1qb5ozt.png?alt=media\&token=67f06856-11df-483d-98a0-ba427faa42cc)

### 调整颜色（Adjust-hue）

原文

> This adjusts the hue value of HSL the same way all of the others do. Again, it takes a percentage value for the change.

翻译

> 通过百分比值的变化，调整HSL的色调值。

```
adjust-hue( $base-color, 20% )
```

![Adjust-hue](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LB1QjQYDebF4yqHF6wh%2F-LB1Vg77mzrHiyF3GZNp%2Fadjust.png?alt=media\&token=e0a93014-bb82-4be2-95c3-b3aa7f403a44)

### 添加Alpha透明度（Adding Alpha Transparency）

原文

> Using our hex color we can do a few things to get it to be a little transparent. We can call hsla, rgba, opacify, and transparentize. All of them accomplish the same thing, just in different ways. I stick to rgba as it comes most naturally to me which takes a color and a value from 0 to 1 for the alpha.

翻译

> 使用我们的十六进制颜色，我们可以做一些事情使它变得有点透明。我们可以调用hsla、rgba、opacify和透明。所有的人都以不同的方式完成同样的事情。我坚持rgba，因为它对我来说是最自然的，它的颜色和值从0到1。

```
rgba( $base-color, .7 )
```

### 色彩和阴影（Tint & Shade）

原文

> Our very own Phil LaPier has added to those base color functions. Both of these are accessible in Bourbon. They mix your color with a value of white (tint) and black (shade) and are similar to Darken and Lighten. They take the color and a % value for the change.

翻译

> 把你的颜色和白色(色调)和黑色(阴影)的值混合，类似于变暗和变亮。它们以颜色和%值进行更改。

```
tint( $base-color, 10% )
shade( $base-color, 10% )
```

![Tint & Shade](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LB1QjQYDebF4yqHF6wh%2F-LB1WpwQiwMHFr9f6Gxi%2Ftint.png?alt=media\&token=690199f7-6d25-4cb7-9386-ff2145301dcc)

### 更多优势（Getting more advanced）

原文

> Once you have those down and you are looking for more control you can look into some more advanced color control with adjust-color, scale-color, change-color. These are for multiple changes in one color function. You can easily lighten the color and add some transparency all in one.

翻译

> 一旦你有了这些，你想要更多的控制，你可以看到一些更高级的颜色控制和调整颜色，标色，改变颜色。这些是一个颜色函数的多个变化。你可以轻松地减轻颜色，并在其中添加一些透明度。

原文

> Some of the best places to use these color functions are for gradients, borders and shadows. When you need a slightly darker border and a slightly lighter inset shadow just adjust a color variable and let Sass do the rest for you. Buttons provide the perfect place to test out the functions. Check out some of the functions used on the thoughtbot buttons:

翻译

> 使用这些颜色函数的一些最好的地方是渐变、边框和阴影。当你需要一个稍微深一点的边框和一个稍微亮一点的嵌入阴影时，只需调整一个颜色变量，让Sass为你做其他的事情。按钮提供了测试函数的最佳位置。查看一些在thoughtbot按钮上使用的功能:

```
border: 1px solid darken($base-color, 20%);
text-shadow: 0 -1px 0 darken($base-color, 10%);
@include box-shadow(inset 0 1px 0 lighten($base-color, 20%));
```

![Getting more advanced](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LB1QjQYDebF4yqHF6wh%2F-LB1XWY_bddRyeZosMyL%2Fadvan.png?alt=media\&token=99bb57b5-287b-4c51-909c-e81ca524065e)

原文出处 <https://robots.thoughtbot.com/controlling-color-with-sass-color-functions>


# Draggable组件库

## Draggable - Shopify出品

{% embed url="<https://shopify.github.io/draggable/examples/>" %}

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LJRS4cc3fv95TQ2gBgz%2F-LJRUV0EXgQQV6QRFig1%2Fimage.png?alt=media\&token=5b4e10a9-0821-417d-a45e-9ec1d198f20a)

## InterActJS

{% embed url="<http://interactjs.io/>" %}

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LJRS4cc3fv95TQ2gBgz%2F-LJRUbLvzWlY5M3ZfcB6%2Fimage.png?alt=media\&token=96b34d01-4765-41bb-a189-70d41c2b0b51)

## GoJS

{% embed url="<https://gojs.net/latest/learn/index.html>" %}

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LJRS4cc3fv95TQ2gBgz%2F-LJRUN1g84AO8J7DUOq0%2Fimage.png?alt=media\&token=b530eb4c-029f-4372-890b-e730ccdd4626)

## jsPlumb

{% embed url="<https://jsplumbtoolkit.com/>" %}

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LJR_fzA7KJVv3Ici1RJ%2F-LJR_s62ftt-kgLWCb2K%2Fimage.png?alt=media\&token=b7749526-1604-4831-ac76-ceb0e6412207)


# Javascript

## 使用技巧

### 生成随机字符串

```javascript
Math.random().toString(32).substring(2)
```


# Ruby


# Ruby on Rails


# 数据库类

## NullDB Connection Adapter

> <https://github.com/nulldb/nulldb>

当我们在本地执行`RAILS_ENV=production rails assets:precompile`编译资源时，会提示我们production数据库不存在，通过Gem `activerecord-nulldb-adapter` 可以避免创建数据库

{% code title="Gemfile" %}

```ruby
gem 'activerecord-nulldb-adapter'
```

{% endcode %}

```bash
RAILS_ENV=production DATABASE_ADAPTER=nulldb rails assets:precompile
```

{% code title="config/database.yml" %}

```yaml
default: &default
  adapter: <%= ENV.fetch('DATABASE_ADAPTER', 'postgresql') %>
  encoding: <%= ENV.fetch('DATABASE_ENCODING', 'unicode') %>
  # For details on connection pooling, see rails configuration guide
  # http://guides.rubyonrails.org/configuring.html#database-pooling
  pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
  host: <%= ENV['DATABASE_HOST'] %>
  username: <%= ENV['DATABASE_USERNAME'] %>
  password: <%= ENV['DATABASE_PASSWORD'] %>
  port: <%= ENV['DATABASE_PORT'] %>
  timeout: 10
```

{% endcode %}


# 日志类

## act-fluent-logger-rails

&#x20;Rails 日志输出到 Fluentd

>

相关文章：

> Collecting and Analyzing Ruby on Rails Logs <https://www.fluentd.org/datasources/rails>


# 价格字段的单位转换

我们在存储一个产品的价格时，字段类型的设计可以有以下几种：

* Decimal - 以元为单位存储浮点数
* Integer - 以分为单位储存整数

如果用浮点数存储数据，那么我们在表单中录入价格时，时常会遇到一个奇怪的现象，录入的价格是`912.00`元，而存储数据库时莫名其妙地变成了`911.999999999999` 。呈现到前端时，就很怪异了。

在ruby中我可以通过下面测试，发现同样问题：

```ruby
2.3.4 :017 > "9.12".to_f * 100
 => 911.9999999999999
```

因此，我们就有了以分为单位存储的想法。但是，以分为单位，我们在生成表单时，又想让用户以元为单位录入价格，我们可能会这么做：

```bash
rails g scaffold product name price_cents:integer
```

{% code title="app/models/product.rb" %}

```ruby
class Product < ApplicationRecord
    # 数据库存储价格字段是 price_cents，整形，单位为分
    def price
        (price_cents || 0) / 100.0
    end

    def price=(v)
        # 这里用.round就是为了解决上面的911.99999999问题
        self.price_cents = (v.to_f * 100).round
    end
end
```

{% endcode %}

{% code title="app/controllers/products\_controller.rb" %}

```ruby
def product_params
    params.require(:product).permit(:name, :price)
end
```

{% endcode %}

{% code title="app/views/products/\_form.html.erb" %}

```markup
  <div class="field">
    <%= form.label :price %>
    <%= form.text_field :price, id: :product_price %>
  </div>
```

{% endcode %}

## 封装一下代码，做到足够DRY

{% code title="app/models/application\_record.rb" %}

```ruby
class ApplicationRecord < ActiveRecord::Base
  self.abstract_class = true

  class << self
    def price_attr(model_attr, db_attr="#{model_attr}_cents")
      class_eval <<-CODE, __FILE__, __LINE__ + 1
        def #{model_attr}
          (#{db_attr} || 0) / 100.0
        end

        def #{model_attr}=(v)
          self.#{db_attr} = (v.to_f * 100).round
        end
      CODE
    end
  end
end
```

{% endcode %}

{% code title="app/models/product.rb" %}

```ruby
class Product < ApplicationRecord
    price_attr :price
end
```

{% endcode %}

以上代码，足够应付我们的价格前后端单位转换问题。如果需要更高级的功能，我们可以尝试Gem [money](https://github.com/RubyMoney/money) 或 [money-rails](https://github.com/RubyMoney/money-rails)

```ruby
require 'money'

# 10.00 USD
money = Money.new(1000, "USD")
money.cents     #=> 1000
money.currency  #=> Currency.new("USD")

# Comparisons
Money.new(1000, "USD") == Money.new(1000, "USD")   #=> true
Money.new(1000, "USD") == Money.new(100, "USD")    #=> false
Money.new(1000, "USD") == Money.new(1000, "EUR")   #=> false
Money.new(1000, "USD") != Money.new(1000, "EUR")   #=> true

# Arithmetic
Money.new(1000, "USD") + Money.new(500, "USD") == Money.new(1500, "USD")
Money.new(1000, "USD") - Money.new(200, "USD") == Money.new(800, "USD")
Money.new(1000, "USD") / 5                     == Money.new(200, "USD")
Money.new(1000, "USD") * 5                     == Money.new(5000, "USD")

# Unit to subunit conversions
Money.from_amount(5, "USD") == Money.new(500, "USD")  # 5 USD
Money.from_amount(5, "JPY") == Money.new(5, "JPY")    # 5 JPY
Money.from_amount(5, "TND") == Money.new(5000, "TND") # 5 TND

# Currency conversions
some_code_to_setup_exchange_rates
Money.new(1000, "USD").exchange_to("EUR") == Money.new(some_value, "EUR")

# Formatting (see Formatting section for more options)
Money.new(100, "USD").format #=> "$1.00"
Money.new(100, "GBP").format #=> "£1.00"
Money.new(100, "EUR").format #=> "€1.00"
```


# 部署

## 部署中遇到的问题

#### 通过capistrano部署代码时，报UglifyJs错误

这个问题，通常是因为UglifyJs不支持ES2015语法，而一些npm包中又有这种新语法，导致编译失败。

临时解决办法：删除UglifyJs插件

{% code title="config/webpack/production.js" %}

```javascript
const environment = require('./environment')
environment.plugins.delete("UglifyJs")
module.exports = environment.toWebpackConfig()
```

{% endcode %}

**或者：**

添加`babel-present-stage-2`, 关闭uglify

```bash
yarn add babel-present-stage-2
```

配置`.babelrc`

```javascript
{
  "presets": [
    ["env", {
      "modules": false,
      "targets": {
        "browsers": "> 1%",
        "uglify": false
      },
      "useBuiltIns": true
    }],
    "stage-2"
  ],
  ...
}
```


# 根据设备类型自动渲染页面

### 添加Gem

```ruby
gem "browser"
```

### 定义方法

```ruby
class ApplicationController < ActionController::Base
  
  before_action :detect_device_variant

  def detect_device_variant
    # 定义手机端调用
    request.variant = :mobile if browser.device.mobile?
    # 定义平板，如果需要
    request.variant = :tablet if browser.device.tablet?
  end

end

```

### 配置View

找到需要配置的view

```ruby
application.html.erb  #原始模板文件
application.html+mobile.erb  #如果定义的有mobile
application.html+tablet.erb  #如果定义的有tablet
```


# 路由

## 配置前后端分离

当我们快速启动一个Rails项目时，rails g scaffold是最好的工具，马上就能生成CRUD页面。但是，CRUD功能往往都是管理端需要，前端最多的还是Index和Show页面。为了实现前后端代码分离，我们把所有前端controller/view放到frontend目录，rails g scaffold生成的作为管理端。配置routes.rb的代码如下：

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LJHcFX8BNxo9me_54xP%2F-LJHyJ0FbrX5zR3yC00m%2Fimage.png?alt=media\&token=1c934fff-208f-4297-b0de-bd2db3e138d4)


# Gems

## 数据库

1. [scenic](https://github.com/thoughtbot/scenic) 待版本控制的数据库视图管理&#x20;
2. [rails\_db](https://github.com/igorkasyanchuk/rails_db) 数据库在线GUI管理
3. [paranoia](https://github.com/rubysherpas/paranoia/) 数据软删除
4. [acts\_as\_list](https://github.com/swanandp/acts_as_list) 实现数据排序
5. [acts-as-taggable-on](https://github.com/mbleigh/acts-as-taggable-on) 给数据打标签
6. [closure\_tree](https://github.com/ClosureTree/closure_tree) 给数据定义树形结构
7. [active\_attr](https://github.com/cgriego/active_attr) 像ActiveRecord一样使用Class，但是不用连接数据库

## 视图

1. [codemirror-rails](https://github.com/fixlr/codemirror-rails/) 代码查看、编辑器
2. [terminal-table](https://github.com/tj/terminal-table/) 可把数组排版为终端显示的表格输出
3. [sanitize](https://github.com/rgrove/sanitize) 清理HTML源代码
4. [kaminari](https://github.com/kaminari/kaminari) Model数据库记录或数组的分页插件&#x20;

## HTTP API

1. [flexirest](https://github.com/flexirest/flexirest/) REST API 转为Model实例
2. &#x20;[faraday](https://github.com/lostisland/faraday) HTTP 客户端

## 待分类

1. [annotate\_models](https://github.com/ctran/annotate_models) 自动为Model、Route、Fixture等生成备注信息

### Kaminari-i18n

分页国际化 <https://rubygems.org/gems/kaminari-i18n>

### Simple\_form

更好用的form <https://github.com/plataformatec/simple_form>

### Closure Tree

ActiveRecord模型层次结构 <https://github.com/ClosureTree/closure_tree>

### Browser

判断浏览器类型 <https://github.com/fnando/browser>

### impressionist

访问统计 <https://github.com/charlotte-ruby/impressionist>

### Rails-ERD

为Rails生成实体关系图 <https://github.com/voormedia/rails-erd>

### RailRoady

为Rails模型和控制器生成UML类图 <https://github.com/preston/railroady>

### Httparty

Http访问 <https://github.com/jnunemaker/httparty>

### RequestStore

全局存储请求 <https://github.com/steveklabnik/request_store>


# Automating your API with JSON Schema

> 来源：<http://tech.degica.com/en/2015/05/02/automating-api-documentation-with-json-schema/>


# 深度冻结变量 Deep Freeze

Ruby自带的`freeze`无法深度冻结，如：

```ruby
$ irb
2.4.4 :001 > h={a: [1,2], b: { c: [2,3]}, c: 'hello'}.freeze
 => {:a=>[1, 2], :b=>{:c=>[2, 3]}, :c=>"hello"}
2.4.4 :002 > h[:c] << ' there'
 => "hello there"
2.4.4 :003 > h
 => {:a=>[1, 2], :b=>{:c=>[2, 3]}, :c=>"hello there"}
```

通过下面扩展，可以实现深度冻结：

```ruby
# Recursively freeze self if it's Enumerable
# Supports all Ruby versions 1.8.* to 2.2.*+
module Kernel
  alias deep_freeze freeze
  alias deep_frozen? frozen?
end

module Enumerable
  def deep_freeze
    if !@deep_frozen
      each(&:deep_freeze)
      @deep_frozen = true
    end
    freeze
  end

  def deep_frozen?
    !!@deep_frozen
  end
end
```

```ruby
2.4.4 :021 > h={a: [1,2], b: { c: [2,3]}, c: 'hello'}.deep_freeze
 => {:a=>[1, 2], :b=>{:c=>[2, 3]}, :c=>"hello"}
2.4.4 :022 > h[:c] << ' there'
RuntimeError: can't modify frozen String
	from (irb):22
	from /Users/xiaohui/.rvm/rubies/ruby-2.4.4/bin/irb:11:in `<main>'
```


# 搭建私有Gem仓库

Gem私有仓库，我们用`Geminabox`快速搭建<https://github.com/geminabox/geminabox>

{% code title="config.ru" %}

```ruby
#
# This is a simple rackup file for geminabox. It allows simple role-based authorization.
#
# roles:
# - developer
# - upload
# - delete
# - admin (can do anything)
#
# For example, a developer who can access the service and upload new gems would have the following roles: `%w(developer upload)
#

require "rubygems"
require "geminabox"

Geminabox.rubygems_proxy = false
Geminabox.data = "/geminabox/data"

API_KEYS = {
  ENV['DEVELOPER_API_KEY'] => { password: '', roles: %w(developer) },
  ENV['ADMIN_API_KEY'] => { password: '', roles: %w(admin) }
}

use Rack::Session::Pool, expire_after: 1000 # sec
use Rack::Protection

Geminabox::Server.helpers do
  def protect!(role='developer')
    unless has_role?(role)
      response['WWW-Authenticate'] = %(Basic realm="Gem In a Box")
      halt 401, "Not Authorized.\n"
    end
  end

  def auth
    @auth ||= Rack::Auth::Basic::Request.new(request.env)
  end

  def username
    auth ? auth.credentials.first : nil
  end

  def password
    auth ? auth.credentials.last : nil
  end

  def user_roles
    API_KEYS[username][:roles]
  end

  def authenticated?
    return false unless auth.provided? && auth.basic? && auth.credentials
    api_key = API_KEYS[username]
    !api_key.nil? && password == api_key[:password]
  end

  def current_user_roles
    authenticated? ? user_roles : []
  end

  def has_role?(role)
    current_user_roles.include?('admin') || current_user_roles.include?(role)
  end
end

Geminabox::Server.before '/upload' do
  protect!('upload')
end

Geminabox::Server.before do
  if request.delete?
    protect!('delete')
  else
    protect!('developer')
  end
end

Geminabox::Server.before '/api/v1/gems' do
  unless env['HTTP_AUTHORIZATION'] == 'API_KEY'
    halt 401, "Access Denied. Api_key invalid or missing.\n"
  end
end

run Geminabox::Server

```

{% endcode %}

把上面内容保存到`config.ru`文件，然后运行`ADMIN_API_KEY=admin rackup`即可启动服务，登录用户名是`admin`，密码填空


# YAML语法

## 数据继承

```ruby
require 'yaml'
y = YAML.load(<<-STR)
.base: &base "Tanmer Inc."
dev:
  name: *base
STR
# => {".base"=>"Tanmer Inc.", "dev"=>{"name"=>"Tanmer Inc."}}
```


# 数据库

## 基于PostgreSQL 的数据库

1. [TimescaleDB](https://www.timescale.com/) 时序数据库
2. [PipelineDB](https://www.pipelinedb.com/) 基于内存存储的流数据库
3. [ZomboDB](https://www.zombodb.com/) 集成ElasticSearch的数据库
4. [Citus](https://www.citusdata.com/) 可水平扩展的分布式数据库，类似PostgreSQL XL

###


# PostgreSQL

## 安装

### Docker安装

1. 我们的项目都会涉及到中文字符，会对中文字符按拼音排序，`Ubuntu`默认安装的`PostgreSQL`是`en_US.UTF-8`字符集排序，要支持拼音培训，需要改为`zh_CN.UTF-8`。
2. Mac下用`brew`安装的PostgreSQL字符集排序是`zn_CN.UTF-8`，但是很奇怪的是他并不能实现拼音排序，经研究之后确实找不到解决办法。\
   因为以上两点，所以我用这个镜像方便我们开发人员快速使用数据库

#### 如何启动

```bash
mkdir -p ~/postgresql/data
docker run -v $(realpath ~/postgresql/data):/var/lib/postgresql/data -p 5432:5432 --name postgresql-10 -d tanmer/postgresql:10
```

#### 进入psql CLI

```
docker exec -it postgresql-10 psql -U postgres
```

#### 测试中文排序是否正确

```
postgres=# select * from (values ('刘少奇'),('刘德华')) as a(c1) order by c1;
  c1
--------
刘德华
刘少奇
(2 rows)

postgres=#
```

#### 镜像 tanmer/postgresql:10 的 Dockerfile内容

```
FROM postgres:10
MAINTAINER Xiaohui <xiaohui@tanmer.com>

RUN sed -i 's!deb.debian.org!mirrors.163.com!' /etc/apt/sources.list \
    && apt update \
    && apt install --reinstall locales \
    && echo zh_CN UTF-8 > /etc/locale.gen \
    && echo zh_CN.UTF-8 UTF-8 >> /etc/locale.gen \
    && echo en_US UTF-8 >> /etc/locale.gen \
    && echo en_US.UTF-8 UTF-8 >> /etc/locale.gen \
    && locale-gen

ENV LC_COLLATE zh_CN.UTF-8

```

### Ubuntu安装

```bash
echo deb http://mirrors.tuna.tsinghua.edu.cn/postgresql/repos/apt/ xenial-pgdg main > /etc/apt/sources.list.d/pgdg.list
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add -
apt-get update
apt-get install postgresql-10
```

更改默认配置

```bash
# 改变数据目录
echo data_directory = \'/data/postgresql/10/main\' > /etc/postgresql/10/main/conf.d/tanmer.conf
# 配置最大连接数
echo max_connections = 1000 >> /etc/postgresql/10/main/conf.d/tanmer.conf
# 允许外部主机连接
echo listen_addresses = \'0.0.0.0\' >> /etc/postgresql/10/main/conf.d/tanmer.conf
# 允许外部用户用密码登录
echo host all all 10.103.0.0/24 md5 >> /etc/postgresql/10/main/pg_hba.conf
# 停止服务
systemctl stop postgresql
# 移动数据目录
mv /var/lib/postgresql /data
# 启动服务
systemctl start postgresql
# 查看服务状态
systemctl status postgresql
```

## 使用

### 创建用户

```sql
create role xiaohui with login password 'this-is-a-password';
```

### 创建数据库

```sql
create database project;
```

### 设置数据库拥有者

```sql
alter database project owner to xiaohui; 
grant all privileges on database project to xiaohui;
```

### 更改数据表的拥有者

```sql
SELECT 'ALTER TABLE '|| schemaname || '.' || tablename ||' OWNER TO new-owner;'
FROM pg_tables WHERE NOT schemaname IN ('pg_catalog', 'information_schema')
ORDER BY schemaname, tablename;

SELECT 'ALTER SEQUENCE '|| sequence_schema || '.' || sequence_name ||' OWNER TO new-owner;'
FROM information_schema.sequences WHERE NOT sequence_schema IN ('pg_catalog', 'information_schema')
ORDER BY sequence_schema, sequence_name;
```

### 清空数据库

执行下面查询，生成对所有表`drop`的SQL，然后复制、粘贴、执行。

```sql
select 'drop table "' || tablename || '" cascade;' as drop_table from pg_tables where schemaname = 'public';
```

## &#x20;查询

### 获取某时区的时间

```sql
// 先获取所有的时区定义
select * from pg_timezone_names order by utc_offset
```

```
               name               | abbrev | utc_offset | is_dst
----------------------------------+--------+------------+--------
 Africa/Abidjan                   | GMT    | 00:00:00   | f
 Africa/Accra                     | GMT    | 00:00:00   | f
 Africa/Addis_Ababa               | EAT    | 03:00:00   | f
 Africa/Algiers                   | CET    | 01:00:00   | f
 Africa/Asmara                    | EAT    | 03:00:00   | f
 Africa/Asmera                    | EAT    | 03:00:00   | f
 Africa/Bamako                    | GMT    | 00:00:00   | f
```

```sql
// 把字段created_at的UTC时间转换为用户设定的本地时间
select created_at, timezone, created_at at time zone 'UTC' at time zone timezone as localtime from users;
```

```
         created_at         |   timezone    |         localtime
----------------------------+---------------+----------------------------
 2018-10-10 03:36:33.686459 | Asia/Shanghai | 2018-10-10 11:36:33.686459
 2018-10-10 04:11:23.707863 | PST           | 2018-10-09 20:11:23.707863
(2 rows)
```


# 基础知识


# PostgreSQL中插入数据

#### 这是Postgresql最短，最简单插入方法，你只需要按顺序的插入指定的值，所以，如果你有10列，你必须指定10个值。

```
-- 假设这里有个 "users" 表只有3列：: first_name, last_name, email, 并且按此顺序排列
users values ('John', 'Doe', 'john@dow.com');
```

#### 如果你有很多列，当时你想指定其中某一列：

```
insert into users (first_name) values ('John');
```

#### 如果想在列中插入JSON数据，只需要将JSON数据包裹在单引号字符串中。

```
insert into users (preferences) values ('{ "beta": true }')
```

#### 如果插入的数据违背了一个特定的约束，这是你可以使用 Postgres 在冲突子句中指定当发生这种情况时该怎么做，例如：想象你有一个webhook系统，你想优雅的处理webhook中重复的事情。

```
-- 如果我们已经记录（捕获）到 webhook 的冲突，就什么都不要做。
insert into stripe_webhooks (event_id)
values ('evt_123')
on conflict do nothing;
```

#### 你也可以在冲突的时候执行 "upserts"   (更新或插入)数据。

```
-- 如果你有一个唯一的email索引
insert into users (email, name)
value ('john@dow.com', 'Jane Doe')
on conflict (email) do update set name = excluded.name; -- excluded.name 指的是 'Jane Doe'
```


# PostgreSQL中更新数据

```
-- 更新 users 表中列名为 updated_at 的所有行数据
update users set updated_at = now();

-- 更新 users 表中列名为 updated_at 条件为 id = 1 这一行数据
update users set updated_at = now() where id = 1;

```


# PostgreSQL中删除数据

```
-- 删除 users 表中条件为id = 1 这条数
delete from users where id = 1;
```


# 数据库管理

## 对表的管理

### 创建表

这里有一个关于创建 "users" 表的例子:

```sql
create table users (
    id serial primary key, -- id 自动递增
    name character varying, -- 指定字符串输出长度大小
    preferences jsonb, -- 字段类型为JSON非常适合存储非结构化的数据
    created_at timestamp without time zone -- 始终以UTC格式存储时间
);
```

你也有机会指定非空约束和默认值：

```sql
create table users (
    id serial primary key,
    name character varying not null,
    active boolean default true
);
```

### 删除表

```sql
drop table funky_users;
```

### 重命名表

```sql
alter table events rename to events_backup;
```

### 清空表

要非常小心这段代码，它会清空 PosgreSQL 表中的所有内容，在开发中很有用处，但是它很少想在生产环境中应用。

```sql
truncate my_table;
```

如果你又一个序列ID列，并且你想重启它的序列（既重启ID重1开始）

```sql
truncate my_table restart identity;
```

### 复制表

有时候表的复制对你很有用：

```sql
create table dupe_users as (select * from users);

-- 这个 'with no data' 意思是只有结构，没有实际的行
create table dupe_users as (select * from users) with no data;
```

### 添加列

这里有一个例子关于在 users 表中添加 created\_at 时间戳列：

```sql
alter table users add column created_at timestamp without time zone;
```

添加一个 String ( varchar ) 类型并且设置非空约束的列：

```sql
alter table users add column bio string character varying not null;
```

添加一个 Boolean 类型并且设置默认值的列：

```sql
alter table users add column active boolean default true;
```


# select  jsonb

例:

```
table_name  -> company_contents
column_name -> content
column_type -> jsonb
data        ->  {
                  "企业类型": '有限公司'，
                  "经营范围": 'IT',
                  "企业资质": {
                    "主板上市": true
                  }
                }

# 查询 ( 企业类型 = '有限公司' ) 的所有企业

select * from company_contens where ( content ->>  '企业类型' = '有限公司' );

# 查询 ( 主板上市 = true ) 的所有企业

select * from company_contents where ( content -> '企业资质' ->> '主板上市' = ture )
```

上面主要运用了 -> 和 ->> 做查询，->  作用是获取 json 对象，->>  则是获取一个文本信息，详细请查阅官方文档: <https://www.postgresql.org/docs/9.6/static/functions-json.html>


# PostgreSQL XL

官网：<https://www.postgres-xl.org/>

官网部署文档：<https://www.postgres-xl.org/documentation/server-start.html>

参考：<https://www.jianshu.com/p/82aaf352b772>

## 概述

Postgres XL，XL是eXtensible Lattice的简写，意识是可扩展的盒子。

### 主要特点：

#### 可伸缩（Scalable）

Postgres XL可实现数据分表存储在多个节点中，或在多节点中同步副本集

#### 完全支持ACID（Fully ACID）

* Atomicity （原子性，或称不可分割性）
* Consistency（一致性）
* Isolation（隔离性，或称独立性）
* Durability（持续性）

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LCwxZlE0BPNhvRZS9iB%2F-LCwyNL-Q_vGgV6IUKE_%2Fimage.png?alt=media\&token=4e9c71ff-e775-4e89-a17e-c2f5c4524527)

目前安装方式只支持二进制安装，rpm或deb等格式以后会支持的

### 组件构成

#### GTM

#### GTM-Proxy

#### Coordinator

#### Datanode

## 安装

这里有3个角色概念：

1. GTM
2. Coordinator
3. Datanode

官方推荐Coordinator和Datanode在同一节点，GTM至少有一个Standby，因此，我们安装一套初始环境，节点做如下安排：

Node1：GTM => 2CPUs + 4G MEM + 20G DISK

Node2：GTM Standby => 2CPUs + 4G MEM + 20G DISK

Node3: coord\_1 + datanode\_1  => 2CPUs + 4G MEM + 200G SSD

Node4: coord\_2 + datanode\_2  => 2CPUs + 4G MEM + 200G SSD

Node5: datanode\_1\_slave => 2CPUs + 4G MEM + 200G SSD

Node6: datanode\_2\_slave => 2CPUs + 4G MEM + 200G SSD

6个节点，2个GTMs，2个Shards, 每个Shard有一个Slave

### 创建postgres用户账号

在每个节点，创建postgres账号，创建存放数据的基础目录

```bash
sudo adduser postgres --disabled-password
sudo mkdir -p /data/pgxc/nodes
sudo chown postgres.postgres /data/pgxc/nodes
```

实现从node1向其他节点ssh免登陆

Node1上：

```bash
sudo su postgres
ssh-keygen
```

一路默认回车，会得到两个文件`/home/postgres/.ssh/id_rsa`和`/home/postgres/.ssh/id_rsa.pub`，把`pub`文件复制到`authorized_keys`中

```bash
cp /home/postgres/.ssh/id_rsa.pub /home/postgres/.ssh/authorized_keys
```

登录Node2-Node6，那`/home/postgres/.ssh/authorized_keys`文件的内容复制到每个节点的`/home/postgres/.ssh/authorized_keys`

从Node1用postgres账号登录Node2-6，确认每个节点都不需要输入密码

### 下载、编译

在每个节点编译安装Postgres-XL，版本为XL9\_5\_R1\_6

```bash
sudo apt-get update
sudo apt-get install build-essential libreadline-dev zlib1g-dev bison flex -y
git clone git://git.postgresql.org/git/postgres-xl.git
cd postgres-xl
git checkout XL9_5_R1_6
./configure
make -j 2
sudo make install
cd contrib
make -j 2
sudo make install
```

编辑`/etc/environment`，添加`/usr/local/pgsql/bin`

### 使用pgxc\_ctl工具初始化集群

在Node1上用postgres账号使用`pgxc_ctl`

```bash
sudo su postgres
pgxc_ctl
# 进入PGXC控制台
PGXC prepare config empty
PGXC exit
```

{% code title="输出：" %}

```
ubuntu@10-103-0-59:~/pgxc_ctl$ pgxc_ctl
/bin/bash
Installing pgxc_ctl_bash script as /home/ubuntu/pgxc_ctl/pgxc_ctl_bash.
Installing pgxc_ctl_bash script as /home/ubuntu/pgxc_ctl/pgxc_ctl_bash.
Reading configuration using /home/ubuntu/pgxc_ctl/pgxc_ctl_bash --home /home/ubuntu/pgxc_ctl --configuration /home/ubuntu/pgxc_ctl/pgxc_ctl.conf
Finished reading configuration.
   ******** PGXC_CTL START ***************

Current directory: /home/ubuntu/pgxc_ctl
PGXC prepare config empty
PGXC exit
```

{% endcode %}

创建`~/pgxc_ctl/pg_hba_extra.conf`文件，这个文件至关重要，把每台服务器的IP都加入信任列表，让每个账户访问数据库都不需要输入密码。否则，在后面的步骤中，创建的新用户将无法访问数据库，即使密码输入正确也不行。

{% code title="pg\_hba\_extra.conf" %}

```
host all all 10.103.0.59/32 trust
host all all 10.103.0.202/32 trust
host all all 10.103.0.97/32 trust
# 其他IP需要输入密码
host all all 0.0.0.0/0 md5
```

{% endcode %}

再次运行`pgxc_ctl`，开始添加集群角色

```bash
# 注：PGXC命令行无法所以用变量
# node1=10.103.0.59
# node3=10.103.0.202
# node5=10.103.0.97
# 添加GTM
pgxc_ctl
PGXC add gtm master gtm1 10.103.0.59 6666 /data/pgxc/nodes/gtm
# 检查GTM状态，这时应该显示的 Running: gtm master
PGXC monitor all
# 添加coordinator
PGXC add coordinator master coord1 10.103.0.202 5432 5433 /data/pgxc/nodes/coord_master none none
PGXC monitor all
# Running: gtm master
# Running: coordinator master coord1
# 添加datanode
PGXC add datanode master dn1 10.103.0.202 5434 5435 /data/pgxc/nodes/datanode none none none
# Running: gtm master
# Running: coordinator master coord1
# Running: datanode master dn1
PGXC add coordinator master coord2 10.103.0.97 5432 5433 /data/pgxc/nodes/coordinator none none
PGXC add datanode master dn2 10.103.0.97 5434 5435 /data/pgxc/nodes/datanode none none none
PGXC monitor all
# Running: gtm master
# Running: coordinator master coord1
# Running: coordinator master coord2
# Running: datanode master dn1
# Running: datanode master dn2
```

现在，1个GTM，2个Coordinator，2个datanode组成了集群

在node1上测试一下：

```
psql -h 10.103.0.202

postgres=# create database test1;
CREATE DATABASE
postgres=# \c test1
You are now connected to database "test1" as user "postgres".
test1=# SELECT * FROM pgxc_node;
 node_name | node_type | node_port |  node_host   | nodeis_primary | nodeis_preferred |   node_id
-----------+-----------+-----------+--------------+----------------+------------------+-------------
 coord1    | C         |      5432 | 10.103.0.202 | f              | f                |  1885696643
 dn1       | D         |      5434 | 10.103.0.202 | f              | f                |  -560021589
 coord2    | C         |      5432 | 10.103.0.97  | f              | f                | -1197102633
 dn2       | D         |      5434 | 10.103.0.97  | f              | f                |   352366662
(4 rows)
```

### 添加GTM

```
PGXC add gtm master gtm1 10.103.0.59 6666 /data/pgxc/nodes/gtm
```

### 添加Coordinator

```
PGXC add coordinator master coord1 10.103.0.202 5432 5433 /data/pgxc/nodes/coordinator none none
```

### 添加Datanode

Master:

```
PGXC add datanode master dn1 10.103.0.97 5434 5435 /data/pgxc/nodes/datanode_master none none none
```

Slave:

```
PGXC add datanode slave dn1 10.103.0.202 5434 5435 /data/pgxc/nodes/datanode_slave none /data/pgxc/nodes/datanode_archlog
```

{% hint style="info" %}
注意：添加slave时，名称，端口都要和master一样
{% endhint %}


# Data Definition

## Table Basics

## Default Values

## Constraints

## System Columns

## Modifying Tables


# 查询技巧

## 聚合

### 窗口函数

<https://www.postgres-xl.org/documentation/tutorial-window.html>

聚合函数可以通过`OVER (PARTITION BY x)`对数据分组，如下例子：计算每个部门的平均工资

```

SELECT depname, empno, salary, avg(salary) OVER (PARTITION BY depname) FROM empsalary;  depname  | empno | salary |          avg          -----------+-------+--------+----------------------- develop   |    11 |   5200 | 5020.0000000000000000 develop   |     7 |   4200 | 5020.0000000000000000 develop   |     9 |   4500 | 5020.0000000000000000 develop   |     8 |   6000 | 5020.0000000000000000 develop   |    10 |   5200 | 5020.0000000000000000 personnel |     5 |   3500 | 3700.0000000000000000 personnel |     2 |   3900 | 3700.0000000000000000 sales     |     3 |   4800 | 4866.6666666666666667 sales     |     1 |   5000 | 4866.6666666666666667 sales     |     4 |   4800 | 4866.6666666666666667(10 rows)
```

还可以通过`OVER (PARTITION BY x ORDER BY y)`对每个分组进行排序，如下例子：对每个部门的工资进行排序

```
SELECT depname, empno, salary,       rank() OVER (PARTITION BY depname ORDER BY salary DESC)FROM empsalary;  depname  | empno | salary | rank -----------+-------+--------+------ develop   |     8 |   6000 |    1 develop   |    10 |   5200 |    2 develop   |    11 |   5200 |    2 develop   |     9 |   4500 |    4 develop   |     7 |   4200 |    5 personnel |     2 |   3900 |    1 personnel |     5 |   3500 |    2 sales     |     1 |   5000 |    1 sales     |     4 |   4800 |    2 sales     |     3 |   4800 |    2(10 rows)
```

`OVER` 中不带`ORDER BY`时，`SUM`函数会汇总所有记录

```

SELECT salary, sum(salary) OVER () FROM empsalary; salary |  sum  --------+-------   5200 | 47100   5000 | 47100   3500 | 47100   4800 | 47100   3900 | 47100   4200 | 47100   4500 | 47100   4800 | 47100   6000 | 47100   5200 | 47100(10 rows)
```

如果`OVER`中带有`ORDER BY`时，`SUM`函数会从上到下逐条汇总

```

SELECT salary, sum(salary) OVER (ORDER BY salary) FROM empsalary; salary |  sum  --------+-------   3500 |  3500   3900 |  7400   4200 | 11600   4500 | 16100   4800 | 25700   4800 | 25700   5000 | 30700   5200 | 41100   5200 | 41100   6000 | 47100(10 rows)
```


# Docker

## 安装

官方文档安装最新版：<https://docs.docker.com/install/linux/docker-ce/ubuntu/#set-up-the-repository>

通过Rancher安装指定版本：<https://rancher.com/docs/rancher/v1.6/en/hosts/#supported-docker-versions>


# Docker日志收集最佳实践

云栖TechDay31期，阿里云容器服务技术专家戒空给大家带来Docker日志收集最佳实践的演讲。本文主要从传统日志处理开始谈起，接下来着重分析Docker日志处理，包括stdout和文件日志，其中还有fluentd-pilot，接着分享了日志存储方案Elasticsearch、graylog2和SLS，最后对正确写日志给出了建议。

以下是精彩内容整理：

传统日志处理

说到日志，我们以前处理日志的方式如下：

* 日志写到本机磁盘上
* 通常仅用于排查线上问题，很少用于数据分析
* 需要时登录到机器上，用grep、awk等工具分析

那么，这种方式有什么缺点呢？

第一，   它的效率非常低，因为每一次要排查问题的时候都要登到机器上去，当有几十台或者是上百台机器的时候，每一台机器去登陆这是一个没办法接受的事情，可能一台机器浪费两分钟，整个几小时就过去了。

第二，   如果要进行一些比较复杂的分析，像grep、awk两个简单的命令不能够满足需求时，就需要运行一些比较复杂的程序进行分析。

第三，   日志本身它的价值不光在于排查一些系统问题上面，可能在一些数据的分析上，可能利用日志来做一些用户的决策，这也是它的价值，如果不能把它利用起来，价值就不能充分的发挥出来。

![0a1a7d979bde3febae235aaaa68cdfcd1498bd4d](https://yqfile.alicdn.com/0a1a7d979bde3febae235aaaa68cdfcd1498bd4d.png)

所以，现在很多公司会采用集中式日志收集的日志处理方式，我们会把日志分布式收集，集中来存储，我们会在所有机器上面把日志都收集起到一个中心，在中心里面做一个日志全文索引搜索，可以通过一个界面去查询，同时这个日志系统后端可以对接一些更复杂的数据处理系统，可以对接监控、报警系统，对接数据挖掘数据分析系统，充分发挥日志的价值。

Docker的日志处理

使用过Docker的人尤其是使用过容器编排系统，比如说我们的容器服务，可能已经注意到这样的一些特点：

容器编排跟传统的布置方式是不一样的，在容器编排里面，资源分配应用跑到哪台机器上面的决策是由容器层来做的，所以你事先不知道你的容器应用会跑到哪台机器上面；还有自动伸缩，根据负载自动增加或者减少容器数量；另外，在整个运行过程中，系统发生一些情况时，比如说你的容器宕掉了，容器服务会自动把容器应用迁到其他的机器上去，整个过程非常动态，如果像传统方式去配制日志的收集工具，从一台机器上面收集某一个应用，在这个动态下面，很难用原来的方式去配置。

基于这些特点，在Docker的日志里面， 我们只能够采用中心化的日志收集方案，你已经没办法再像原来登到一台机器上面去看它的日志是什么，因为你不知道它其实在哪个机器上面。

**stdout和文件日志**

Docker的日志我们可以把它分成两类，一类是stdout标准输出，另外一类是文件日志。stdout是写在标准输出里面的日志，比如你在程序里面，通过print或者echo来输出的时候，这种输出标准在linux上面其实是往一个ID为零的文件表述书里面去写；另外的就是文件日志，文件日志就是写在磁盘上的日志，一般来说我们会在传统的应用里面会用得多一些。

**stdout**

![084d5cf764d8f6c8265e8db27462eb4945a30f85](https://yqfile.alicdn.com/084d5cf764d8f6c8265e8db27462eb4945a30f85.png)

在Docker的场景里面，目前比较推崇这种标准输出的日志，标准输出日志具体过程如图。标准输出日志的原理在于，当在启动进程的时候，进程之间有一个父子关系，父进程可以拿到子进程的标准输出。拿到子进程标准输出的后，父进程可以对标准输出做所有希望的处理。

![907576eca72644da17bae4fbff25c687b1a905ab](https://yqfile.alicdn.com/907576eca72644da17bae4fbff25c687b1a905ab.png)

例如，我们通过exec.Command启动了一个命令，带一些参数，然后就可以通过标准的pipeline拿到标准输出，后面就可以拿到程序运行过程中产生标准输出。 Docker也是用这个原理来拿的，所有的容器通过Docker Daemon启动，实际上属于Docker的一个子进程， 它可以拿到你的容器里面进程的标准输出，然后拿到标准输出之后，会通过它自身的一个叫做LogDriver的模块来处理，LogDriver就是Docker用来处理容器标准输出的一个模块。 Docker支持很多种不同的处理方式，比如你的标准输出之后，在某一种情况下会把它写到一个日志里面，Docker默认的JSON File日志，除此之外，Docker还可以把它发送到syslog里面，或者是发送到journald里面去，或者是gelf的一个系统。

怎么配置log driver呢？

用Docker来启动容器的话，你有两种方式来配置LogDriver：

![ad484d1d85a26fbcb23f697375184251f7ae0803](https://yqfile.alicdn.com/ad484d1d85a26fbcb23f697375184251f7ae0803.png)

第一种方式是在Daemon上配置，对所有的容器生效。你配置之后，所有的容器启动，如果没有额外的其他配制，默认情况下就会把所有容器标准输出全部都发送给Syslog服务，这样就可以在这个Syslog服务上面收集这台机器上的所有容器的标准输出；

![217c388c333cbc52206cd99bbbbd6eaa5fd95ec1](https://yqfile.alicdn.com/217c388c333cbc52206cd99bbbbd6eaa5fd95ec1.png)

第二种方式是在容器上配置，只对当前容器生效。如果你希望这个配置只对一个容器生效，不希望所有容器都受到影响，你可以在容器上面配置。启动一个容器，单独配置它自身使用的logdriver。

![bd65c736f8d8b62b9eb7a762957365a25440ceee](https://yqfile.alicdn.com/bd65c736f8d8b62b9eb7a762957365a25440ceee.png)

其实Docker之前已经支持了很多的logdriver，图中列表是直接从Docker的官方文档上面拿到的。

**文件日志**

对于stdout的这种日志，在Docker里面现在处理起来还是比较方便的，如果没有现成Logdriver的也可以自己实现一个，但是对于文件日志处理起来就没有这么简单了。如果在一个容器里面写了日志，文件位于容器内部，从宿主机上无法访问，的确你是可以根据Docker用的devicemapper、overlayfs访问到它里面的一个文件，但是这种方式跟Docker的实现机制是有关系的，将来它如果改变，你的方案就失效了；另外，容器运行非常动态，日志收集程序难以配置，如果有一个日志收集的程序，在机器上面配置要收集哪个文件，它的格式是什么样子的、发送到哪儿？因为一台机器上面容器是一直在动态变的，它随时可能在增加一个或者删除一个，事先你并不知道这台机器上会跑了多少个容器，他们的配置是怎么样子的，他们的日志是写在哪儿的，所以没办法预先在一台机器上面把这个采集程序配好，这就是文件收集比较难的两个地方。

最简单的一个方案，给每个容器弄一个日志采集进程，这个进程跑到容器里面，就可以解决以上的两个问题，第一因为它跑到容器里面，就可以访问到容器里面所有的文件，包括日志文件；第二它跟容器在一起，当容器启动的时候，收集日志的进程也启动了，当容器销毁的时候，进程也就被销毁掉了。

这个方案非常简单，但是其实会有很多的缺点：

第一，   因为每个容器都有一个日志的进程，意味着你的机器上面有100个容器，就需要启动一百个日志设备的程序，资源的浪费非常厉害。

第二，   在做镜像的时候，需要把容器里面日志采集程序做到镜像里面去，对你的镜像其实是有入侵的，为了日志采集，不得不把自己的日志程序再做个新镜像，然后把东西放进去，所以对你的镜像过程是有入侵性的。

第三，   当一个容器里面好多个进程的时候，对于容器的资源管理，会干扰你对容器的资源使用的判断，包括对于在做资源分配和监控的时候，都会有一些这样的干扰。

**fluentd-pilot**

在容器服务上面，我们新开发了一个工具，称之为fluentd-pilot。

fluentd-pilot是一个开源的日志采集工具，适合直接在一台机器上面跑单个进程模式。fluentd-pilot有这样的一些特点：

* 一个单独fluentd进程，收集机器上所有容器的日志。不需要为每个容器启动一个fluentd进程；
* 声明式配置。使用label声明要收集的日志文件的路径；
* 支持文件和stdout；
* 支持多种后端存储：elasticsearch, 阿里云日志服务, graylog2…

![24127443e6b05f055952dd41bc476b8b2a751d06](https://yqfile.alicdn.com/24127443e6b05f055952dd41bc476b8b2a751d06.png)

具体是怎么做呢？如图，这是一个简单的结构，在Docker宿主机上面部署一个fluentd-pilot容器，然后在容器里面启动的时候，我们要声明容器的日志信息，fluentd-pilot会自动感知所有容器的配置。每次启动容器或者删除容器的时候，它能够看得到，当看到容器有新容器产生之后，它就会自动给新容器按照你的配置生成对应的配置文件，然后去采集，最后采集回来的日志同样也会根据配置发送到后端存储里面去，这里面后端主要指的elasticsearch或者是SLS这样的系统，接下来你可以在这个系统上面用一些工具来查询等等。整个这一块在Docker宿主机上面，外面的就是外部系统，由这两个部分来组成。

&#x20;                              docker run -d \\

&#x20;                                                         -v /var/run/docker.sock:/var/run/docker.sock \\

&#x20;                                                         -v /:/host \\

&#x20;                                                         -e FLUENTD\_OUTPUT=elasticsearch \\

&#x20;                                                         -e ELASTICSEARCH\_HOST=${ELASTICSEARCH\_HOST} \\

&#x20;                                                         -e ELASTICSEARCH\_PORT=${ELASTICSEARCH\_PORT} \\

&#x20;                                                         registry.cn-hangzhou.aliyuncs.com/acs-sample/fluentd-pilot:0.1

我们既然要用fluentd-pilot，就得先把它启动起来。还要有一个日志系统，日志要集中收集，必然要有一个中间服务去收集和存储，所以要先把这种东西准备好，然后我们在每一个收集日志的机器上面部署一个fluentd-pilot，用这个命令来部署，其实现在它是一个标准的Docker镜像，内部支持一些后端存储，可以通过环境变量来指定日志放到哪儿去，这样的配置方式会把所有的收集到的日志全部都发送到elasticsearch里面去，当然两个管挂载是需要的，因为它连接Docker，要感知到Docker里面所有容器的变化，它要通过这种方式来访问宿主机的一些信息。

&#x20;                                              docker run -it --rm -p 10080:8080 \\

&#x20;                                                          -v /usr/local/tomcat/logs \\

&#x20;                                                          \--label aliyun.logs.catalina=stdout \\

&#x20;                                                          \--label aliyun.logs.access=/usr/local/tomcat/logs/localhost\_access\_log.\*.txt \                                              tomcat

配置好之后启动应用，我们看应用上面要收集的日志，我该在上面做什么样的声明？关键的配置有两个，一是label catalina，声明的时候要收集容器的日志，所有的名字都可以；二是声明access，这也是个名字，都可以用你喜欢的名字。这样一个路径的地址，当你通过这样的配置来去启动fluentd-pilot容器之后，它就能够感觉到这样一个容器的启动事件，它会去看容器的配置是什么，要收集这个目录下面的文件日志，然后告诉fluentd-pilot去中心配置并且去采集，这里有一个-V，实际上跟Logs是一致的，在容器外面实际上没有一种通用的方式能够获取到容器里面的文件，所有我们主动把目录从宿主机上挂载进来，这样就可以在宿主机上看到目录下面所有的东西。

除了最简单的场景之外，你的日志可能会有一些更复杂的特性，比如你的日志格式是什么样子，你可能希望在收集之后加一些内容，便于搜索，当你在真用的时候，它不光是一个非常简单的容器，它可能属于某一个业务或者属于某一个应用，那么，你希望在收集的时候能够有一些关联信息，所以你可以指定日志格式是什么样子，然后可以在日志里添加tag，这些tag相当于一些关键信息，可以附加任何需要的关联信息，这样将来在搜索的时候可以更方便的把这些日志聚在一块；而且，它可以指定很多的后端，fluetnd-pilot支持多种后端，使用环境变量FLUENTD\_OUTPUT指定后端类型。

![31356902fce27c81f25f4ed12ebaf559801526f9](https://yqfile.alicdn.com/31356902fce27c81f25f4ed12ebaf559801526f9.png)

fluent-pilot已经开源，如果功能不满足需求，可以自己定制，自己修改代码实现需要的功能。它的结构比较简单，有这样几个模块：

最上层是容器事件管理，这一块跟Docker进行交互，它会感知Docker的创建容器，然后做出相应的生成配置或者清理配置上的事情；解析容器label跟容器配置，当你创建一个新容器之后，就会用这个模块拿到新容器的配置，然后生成对应的配置文件；FluentdController主要是用来维护对应进程，包括控制什么时候加载新配置，然后检测一些健康状态等等；再下面就是Fluentd的一些插件，如果你需要增加一些日志的后端，就可以自己实现一些插件，放在这个里面，然后再生成跟对应的插件相关的一些配置。  &#x20;

**fleuntd-pilot+阿里云容器服务**

以上是fleuntd-pilot本身的一些能力，现在你可以在任何地方使用它，但是在容器服务上面我们针对它做了一些更加灵活方便、更酷的一些事情，容器服务为fluentd-pilot进行优化：

第一，   自动识别aliyun.logs标签，并创建Volume；

第二，   重新部署，新容器自动复用已有的Volume，避免日志丢失。

**原生支持SLS**

容器服务有一个很棒的特点，它会跟其他的云产品做一些非常方便的集成，这对于用户来说，在使用容器服务的时候，云产品能够更加方便的使用。比如说在日志方面，阿里云容器服务专为SLS做了优化，让用户更简单的在容器服务上使用SLS。SLS是阿里云提供的日志服务，性能强悍，使用方便，还可以对接ODPS等数据系统；支撑1W台物理机，一天12TB日志数据，IOPS>= 2W，采集平均<1 S;单机：在1个CPU core情况下，可以实时采集15-18MB/S  日志量；如果配置中增加可以用线程数目，可水平扩展；是阿里云环境下的最佳日志方案。

![31e5b3c8a14b9b823e90e490374a3586715ab2dd](https://yqfile.alicdn.com/31e5b3c8a14b9b823e90e490374a3586715ab2dd.png)

优化优点具体体现在：自动创建sls的project, logstore；同时支持stdout和文件日志，使用同样的方式配置。

**容器服务日志方案**

![a19e459a0407275506ab167a81c838c1da492ec3](https://yqfile.alicdn.com/a19e459a0407275506ab167a81c838c1da492ec3.png)

比如在容器服务上面布一个tomcat，可能会写如图的一个标准Docker的模板。 &#x20;

当你通过部署之后，就可以在日志服务上面看到会生成两个东西，都是创建好的，加一些前缀来区分，不用管一些配置，你唯一要做的事是什么呢？点日志索引查询，然后到日志搜索界面，刚才启动的时候一些日志，可以看到从哪过来的，这条日志的内容是什么，这些信息都已经很快的出现了，包括需要检索可以选中一个时间段，输入关健词去做一些搜索，自动在日志服务上创建并配置logstore。

![f8d8cf921aa5d5b9c86f78ad5a0f205a73a16f17](https://yqfile.alicdn.com/f8d8cf921aa5d5b9c86f78ad5a0f205a73a16f17.png)

![81a86c70ae9d856a86f17ad0aae3387b9afdbfdc](https://yqfile.alicdn.com/81a86c70ae9d856a86f17ad0aae3387b9afdbfdc.png)

我们在容器服务上面做到了对于文件日志的收集，并且方式也都非常简单，都是你可以设立一个label，通过label方式可以收集到所有的日志。

**日志存储方案**

![83cd5bf772f06482cfb6a3dded2442a74f794d48](https://yqfile.alicdn.com/83cd5bf772f06482cfb6a3dded2442a74f794d48.png)

最后简单的介绍几种日志存储方案的对比，图为现在比较流行的日志方案Elasticsearch，它本身并不提供界面，ELK中的E，基于Lucene，主要用于日志索引、存储和分析；通常配合Kibana展示日志，免费，支持集群模式，可以搭一个Docker用的生产环境可用的系统。

![673ce8addde3fc88bd56fb5d818f5d3a999104eb](https://yqfile.alicdn.com/673ce8addde3fc88bd56fb5d818f5d3a999104eb.png)

接下来是graylog2，目前不算很流行，但是也是功能很强大的系统，它不像Elasticsearch还需要配合其它去使用，它自身拥有日志存储、索引以及展示所有的功能，都在一个系统里面实现，它可以设置一些报警规则，当日志里面出现一些关键字的时候自动报警，这个功能还是非常有用的，免费、支持集群模式，可以在生产环境里面搭一个Docker用的生产系统。

![42b6ee688a1689dcbe90622a6a241d9265512c98](https://yqfile.alicdn.com/42b6ee688a1689dcbe90622a6a241d9265512c98.png)

阿里云的日志服务SLS特点如下：

* 阿里云托管，不需要自己维护
* 支持多用户和权限管理
* 可以对接ODPS等系统
* 支撑1W台物理机，一天12TB日志数据，IOPS>= 2W，采集平均<1 S;单机：在1个CPU core情况下，可以实时采集15-18MB/S  日志量；如果配置中增加可以用线程数目，可水平扩展

用正确的方式写日志

那么，我们怎么样去收集日志、存储日志，用什么样的系统，日志的源头和写日志我们又该怎么来做，有这样几个建议：

1\.         选择合适的日志框架，不要直接print；

2\.         为每一条日志选择正确的level，该debug的不要用info；

3\.         附加更多的上下文信息；

4\.         使用json、csv等日志格式，方便工具解析；

5\.         尽量不要使用多行日志(Java Exception Stack)。


# Harbor搭建私有镜像服务

官方文档：<https://github.com/vmware/harbor>

相关文章：

<https://juejin.im/post/5a52f858f265da3e4e2577fd>


# Kubernetes

官网 <https://kubernetes.io/>

#### [Kubernetes](https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/) is an open-source system for automating deployment, scaling, and management of containerized applications.

{% content-ref url="/pages/-LAHScHECR2Y2xy8pGyG" %}
[参考资源](/kubernetes/references)
{% endcontent-ref %}

{% content-ref url="/pages/-LAHuqfT9mjcxsHQ2Xl1" %}
[工具](/kubernetes/gong-ju)
{% endcontent-ref %}

{% content-ref url="/pages/-LAI00uSb6ue8nwq4cb4" %}
[Rancher方式安装Kubernetes](/kubernetes/tong-guo-rancher-guan-li-kubernetes)
{% endcontent-ref %}


# 参考资源

### 文档

* 官方文档 <https://kubernetes.io/docs/home/?path=browse>
* Jimmy Song Kubernetes 手册 <https://jimmysong.io/kubernetes-handbook/>


# Kubeadm方式安装Kubernetes

官方文档：<https://kubernetes.io/docs/setup/independent/install-kubeadm/>

## 安装Docker 1.12

```bash
sudo -s
curl https://releases.rancher.com/install-docker/1.12.sh | sh
```

脚本来自Rancher <https://rancher.com/docs/rancher/v1.6/en/hosts/#supported-docker-versions>

### 配置Docker

修改为国内源，修改数据目录为/data/docker

{% code title="/etc/docker/daemon.json" %}

```javascript
{
    "registry-mirrors": ["https://registry.docker-cn.com"],
    "storage-driver": "overlay2",
    "graph": "/data/docker"
}
```

{% endcode %}

{% hint style="warning" %}
注意：如果\`storage-driver\`是\`aufs\`，那么参数\`graph\`无法使用，会导致docker重启失败。从docker CE版本开始，storage driver就推荐使用\`overlay2\`
{% endhint %}

## 安装kubeadm和kubectl

这里需要翻墙，参考 [Ubuntu](https://doc.tanmer.cn/ubuntu) 中的翻墙技巧

```bash
apt-get update && apt-get install -y apt-transport-https curl
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
cat <<EOF >/etc/apt/sources.list.d/kubernetes.list
deb http://apt.kubernetes.io/ kubernetes-xenial main
EOF
apt-get update
apt-get install -y kubelet kubeadm kubectl
```

检查`Docker`是否使用`Cgroup Driver`

```bash
docker info | grep -i cgroup
cat /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
```

为了让docker和k8s的Cgroup Driver保持一致，在参数后面加上 `--cgroup-driver=cgroupfs`

## 创建Master节点

创建之前，先关闭swap

```bash
sudo swapoff -a
# 注释掉swap分区
sudo vi /etc/fstab
```

### 预初始化集群

{% hint style="info" %}
这里说预初始化集群，是因为这次不是真正的初始化，只是借助他找出依赖的Docker镜像
{% endhint %}

因为k8s的镜像都在Google服务器行，我们不翻墙是无法访问的。

```bash
sudo kubeadm init --apiserver-cert-extra-sans=k8s-api.prod
```

这里`--apiserver-cert-extra-sans=k8s-api.prod`是为了以后支持高可用master集群，统一一个域名，省去后期麻烦。

### 监视Docker日志

执行之后，新开窗口，监控docker日志，记录哪些镜像下载失败，并记录下来

```bash
journalctl -u docker -f
```

输出大概是这样的

```
Apr 21 21:40:25 node1 dockerd[1144]: time="2018-04-21T21:40:25.098887386+08:00" level=error msg="Handler for GET /v1.24/images/k8s.gcr.io/pause-amd64:3.1/json returned error: No such image: k8s.gcr.io/pause-amd64:3.1"
Apr 21 21:40:28 node1 dockerd[1144]: time="2018-04-21T21:40:28.515676456+08:00" level=error msg="Handler for GET /v1.24/images/k8s.gcr.io/pause-amd64:3.1/json returned error: No such image: k8s.gcr.io/pause-amd64:3.1"
Apr 21 21:40:28 node1 dockerd[1144]: time="2018-04-21T21:40:28.528631577+08:00" level=error msg="Handler for GET /v1.24/images/k8s.gcr.io/pause-amd64:3.1/json returned error: No such image: k8s.gcr.io/pause-amd64:3.1"
Apr 21 21:40:28 node1 dockerd[1144]: time="2018-04-21T21:40:28.535282319+08:00" level=error msg="Handler for GET /v1.24/images/k8s.gcr.io/pause-amd64:3.1/json returned error: No such image: k8s.gcr.io/pause-amd64:3.1"
Apr 21 21:40:28 node1 dockerd[1144]: time="2018-04-21T21:40:28.544880690+08:00" level=error msg="Handler for GET /v1.24/images/k8s.gcr.io/pause-amd64:3.1/json returned error: No such image: k8s.gcr.io/pause-amd64:3.1"
```

{% hint style="info" %}
这个窗口不要关，后面还有好几个镜像需要手动从国外下载。
{% endhint %}

回到刚才的`kubeadm init`窗口，按⌃c终止任务，然后通过如下命令找出需要下载的Docker镜像

```
root@node1:/etc# grep image /etc/kubernetes/manifests/*
/etc/kubernetes/manifests/etcd.yaml:    image: k8s.gcr.io/etcd-amd64:3.1.12
/etc/kubernetes/manifests/kube-apiserver.yaml:    image: k8s.gcr.io/kube-apiserver-amd64:v1.10.1
/etc/kubernetes/manifests/kube-controller-manager.yaml:    image: k8s.gcr.io/kube-controller-manager-amd64:v1.10.1
/etc/kubernetes/manifests/kube-scheduler.yaml:    image: k8s.gcr.io/kube-scheduler-amd64:v1.10.1
```

目前，我们就找出了如下5个用到的镜像：

```
k8s.gcr.io/pause-amd64:3.1
k8s.gcr.io/etcd-amd64:3.1.12
k8s.gcr.io/kube-apiserver-amd64:v1.10.1
k8s.gcr.io/kube-controller-manager-amd64:v1.10.1
k8s.gcr.io/kube-scheduler-amd64:v1.10.1
```

现在，找一台国外的服务器，加载上面的镜像，推送到国内自己的Gitlab Docker Registry

```bash
# 从国外下载镜像

images=(
k8s.gcr.io/pause-amd64:3.1
k8s.gcr.io/etcd-amd64:3.1.12
k8s.gcr.io/kube-apiserver-amd64:v1.10.1
k8s.gcr.io/kube-controller-manager-amd64:v1.10.1
k8s.gcr.io/kube-scheduler-amd64:v1.10.1
k8s.gcr.io/kube-proxy-amd64:v1.10.1
)

for sourceImageName in ${images[@]} ; do
    targetImageName=docker.corp.tanmer.com/tanmer/dockers/${sourceImageName}
    (    docker pull ${sourceImageName} \
      && docker tag ${sourceImageName} ${targetImageName} \
      && docker push ${targetImageName} \
      && docker rmi ${targetImageName} \
    ) || break
done
```

现在，我们回到刚才要安装`kubeadm init`的服务器，执行下面命令，下载镜像

```bash
# 恢复镜像到国内

images=(
k8s.gcr.io/pause-amd64:3.1
k8s.gcr.io/etcd-amd64:3.1.12
k8s.gcr.io/kube-apiserver-amd64:v1.10.1
k8s.gcr.io/kube-controller-manager-amd64:v1.10.1
k8s.gcr.io/kube-scheduler-amd64:v1.10.1
k8s.gcr.io/kube-proxy-amd64:v1.10.1
)

for targetImageName in ${images[@]} ; do
    sourceImageName=docker.corp.tanmer.com/tanmer/dockers/${targetImageName}
    (    docker pull ${sourceImageName} \
      && docker tag ${sourceImageName} ${targetImageName} \
      && docker rmi ${sourceImageName} \
    ) || break
done
```

### 正式初始化集群

```bash
# 重置刚才的集群命令
kubeadm reset
# 重新初始化
kubeadm init --pod-network-cidr=10.244.0.0/16 \
             --apiserver-cert-extra-sans=k8s-api.prod \
             --kubernetes-version 1.10.1
```

{% hint style="info" %}
\--pod-network-cidr=10.244.0.0/16 是因为我们使用Flannel网络，会在后面配置。

\--apiserver-cert-extra-sans=k8s-api.prod 是为了以后支持高可用master集群，统一一个域名，省去后期麻烦。

\--kubernetes-version=1.10.1 指定k8s版本

{% endhint %}

初始化成功之后，输入如下：

```
root@node1:~# kubeadm init --pod-network-cidr=10.244.0.0/16
[init] Using Kubernetes version: v1.10.1
[init] Using Authorization modes: [Node RBAC]
[preflight] Running pre-flight checks.
	[WARNING FileExisting-crictl]: crictl not found in system path
Suggestion: go get github.com/kubernetes-incubator/cri-tools/cmd/crictl
[preflight] Starting the kubelet service
[certificates] Generated ca certificate and key.
[certificates] Generated apiserver certificate and key.
[certificates] apiserver serving cert is signed for DNS names [node1 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 10.0.1.10]
[certificates] Generated apiserver-kubelet-client certificate and key.
[certificates] Generated etcd/ca certificate and key.
[certificates] Generated etcd/server certificate and key.
[certificates] etcd/server serving cert is signed for DNS names [localhost] and IPs [127.0.0.1]
[certificates] Generated etcd/peer certificate and key.
[certificates] etcd/peer serving cert is signed for DNS names [node1] and IPs [10.0.1.10]
[certificates] Generated etcd/healthcheck-client certificate and key.
[certificates] Generated apiserver-etcd-client certificate and key.
[certificates] Generated sa key and public key.
[certificates] Generated front-proxy-ca certificate and key.
[certificates] Generated front-proxy-client certificate and key.
[certificates] Valid certificates and keys now exist in "/etc/kubernetes/pki"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/admin.conf"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/kubelet.conf"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/controller-manager.conf"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/scheduler.conf"
[controlplane] Wrote Static Pod manifest for component kube-apiserver to "/etc/kubernetes/manifests/kube-apiserver.yaml"
[controlplane] Wrote Static Pod manifest for component kube-controller-manager to "/etc/kubernetes/manifests/kube-controller-manager.yaml"
[controlplane] Wrote Static Pod manifest for component kube-scheduler to "/etc/kubernetes/manifests/kube-scheduler.yaml"
[etcd] Wrote Static Pod manifest for a local etcd instance to "/etc/kubernetes/manifests/etcd.yaml"
[init] Waiting for the kubelet to boot up the control plane as Static Pods from directory "/etc/kubernetes/manifests".
[init] This might take a minute or longer if the control plane images have to be pulled.
[apiclient] All control plane components are healthy after 21.002070 seconds
[uploadconfig] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[markmaster] Will mark node node1 as master by adding a label and a taint
[markmaster] Master node1 tainted and labelled with key/value: node-role.kubernetes.io/master=""
[bootstraptoken] Using token: sxs6qp.neypsgvrkx4whmqm
[bootstraptoken] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstraptoken] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstraptoken] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstraptoken] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[addons] Applied essential addon: kube-dns
[addons] Applied essential addon: kube-proxy

Your Kubernetes master has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

You can now join any number of machines by running the following on each node
as root:

  kubeadm join 10.0.1.10:6443 --token sxs6qp.neypsgvrkx4whmqm --discovery-token-ca-cert-hash sha256:242ca9d395b945ca774b28dbfc617f250152ae0a9b700c55be682f6f35a53edd

```

根据输出提示，我们拷贝kube配置文件，测试通过`kubectl`获取集群状态。

```bash
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

xiaohui@node1:~$ kubectl cluster-info
Kubernetes master is running at https://10.0.1.10:6443
KubeDNS is running at https://10.0.1.10:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
```

## 加入worker节点

### 安装Docker、kubeadm和kubectl

登录`node2`服务器，按照上面步骤安装`Docker`、`kubeadm`和`kubectl`

执行Master节点的输出提示：

```bash
kubeadm join 10.0.1.10:6443 --token sxs6qp.neypsgvrkx4whmqm --discovery-token-ca-cert-hash sha256:242ca9d395b945ca774b28dbfc617f250152ae0a9b700c55be682f6f35a53edd
```

这里也可以用journalctl -u docker -f查看日志，知道哪些镜像下载失败。

### 下载镜像

```bash
images=(
k8s.gcr.io/pause-amd64:3.1
k8s.gcr.io/kube-proxy-amd64:v1.10.1
)

for targetImageName in ${images[@]} ; do
    sourceImageName=docker.corp.tanmer.com/tanmer/dockers/${targetImageName}
    (    docker pull ${sourceImageName} \
      && docker tag ${sourceImageName} ${targetImageName} \
      && docker rmi ${sourceImageName} \
    ) || break
done
```

## 安装Flannel网络

现在，我们回到Master节点，查看节点状态，会发现他们的状态都是NotReady，这是因为我们还没有安装网络插件。

{% code title="On node1" %}

```bash
xiaohui@node1:~$ kubectl get no
NAME      STATUS     ROLES     AGE       VERSION
node1     NotReady   master    23m       v1.10.1
node2     NotReady   <none>    5m        v1.10.1
```

{% endcode %}

继续翻墙下载镜像

```bash
# 从国外下载镜像

images=(
k8s.gcr.io/k8s-dns-sidecar-amd64:1.14.8
k8s.gcr.io/k8s-dns-dnsmasq-nanny-amd64:1.14.8
k8s.gcr.io/k8s-dns-kube-dns-amd64:1.14.8
)

for sourceImageName in ${images[@]} ; do
    targetImageName=docker.corp.tanmer.com/tanmer/dockers/${sourceImageName}
    (    docker pull ${sourceImageName} \
      && docker tag ${sourceImageName} ${targetImageName} \
      && docker push ${targetImageName} \
      && docker rmi ${targetImageName} \
    ) || break
done

# 恢复镜像到国内

images=(
k8s.gcr.io/k8s-dns-sidecar-amd64:1.14.8
k8s.gcr.io/k8s-dns-dnsmasq-nanny-amd64:1.14.8
k8s.gcr.io/k8s-dns-kube-dns-amd64:1.14.8
)

for targetImageName in ${images[@]} ; do
    sourceImageName=docker.corp.tanmer.com/tanmer/dockers/${targetImageName}
    (    docker pull ${sourceImageName} \
      && docker tag ${sourceImageName} ${targetImageName} \
      && docker rmi ${sourceImageName} \
    ) || break
done

# 从国外下载镜像

docker pull quay.io/coreos/flannel:v0.9.1-amd64
docker tag quay.io/coreos/flannel:v0.9.1-amd64 docker.corp.tanmer.com/tanmer/dockers/flannel:v0.9.1-amd64
docker push docker.corp.tanmer.com/tanmer/dockers/flannel:v0.9.1-amd64
docker rmi docker.corp.tanmer.com/tanmer/dockers/flannel:v0.9.1-amd64

# 恢复镜像到国内

docker pull docker.corp.tanmer.com/tanmer/dockers/flannel:v0.9.1-amd64
docker tag docker.corp.tanmer.com/tanmer/dockers/flannel:v0.9.1-amd64 quay.io/coreos/flannel:v0.9.1-amd64
docker rmi docker.corp.tanmer.com/tanmer/dockers/flannel:v0.9.1-amd64

```

### 开始安装

```yaml
cat <<EOS|kubectl apply -f -
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: flannel
rules:
  - apiGroups:
      - ""
    resources:
      - pods
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - nodes
    verbs:
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - nodes/status
    verbs:
      - patch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: flannel
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: flannel
subjects:
- kind: ServiceAccount
  name: flannel
  namespace: kube-system
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: flannel
  namespace: kube-system
---
kind: ConfigMap
apiVersion: v1
metadata:
  name: kube-flannel-cfg
  namespace: kube-system
  labels:
    tier: node
    app: flannel
data:
  cni-conf.json: |
    {
      "name": "cbr0",
      "type": "flannel",
      "delegate": {
        "isDefaultGateway": true
      }
    }
  net-conf.json: |
    {
      "Network": "10.244.0.0/16",
      "Backend": {
        "Type": "vxlan"
      }
    }
---
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  name: kube-flannel-ds
  namespace: kube-system
  labels:
    tier: node
    app: flannel
spec:
  template:
    metadata:
      labels:
        tier: node
        app: flannel
    spec:
      hostNetwork: true
      nodeSelector:
        beta.kubernetes.io/arch: amd64
      tolerations:
      - key: node-role.kubernetes.io/master
        operator: Exists
        effect: NoSchedule
      serviceAccountName: flannel
      initContainers:
      - name: install-cni
        image: quay.io/coreos/flannel:v0.9.1-amd64
        command:
        - cp
        args:
        - -f
        - /etc/kube-flannel/cni-conf.json
        - /etc/cni/net.d/10-flannel.conf
        volumeMounts:
        - name: cni
          mountPath: /etc/cni/net.d
        - name: flannel-cfg
          mountPath: /etc/kube-flannel/
      containers:
      - name: kube-flannel
        image: quay.io/coreos/flannel:v0.9.1-amd64
        command: [ "/opt/bin/flanneld", "--ip-masq", "--kube-subnet-mgr" ]
        securityContext:
          privileged: true
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        volumeMounts:
        - name: run
          mountPath: /run
        - name: flannel-cfg
          mountPath: /etc/kube-flannel/
      volumes:
        - name: run
          hostPath:
            path: /run
        - name: cni
          hostPath:
            path: /etc/cni/net.d
        - name: flannel-cfg
          configMap:
            name: kube-flannel-cfg
EOS
```

现在查看集群状态，所有`STATUS`都变成了`Ready`

```bash
xiaohui@node1:~$ kubectl get no
NAME      STATUS    ROLES     AGE       VERSION
node1     Ready     master    51m       v1.10.1
node2     Ready     <none>    34m       v1.10.1

xiaohui@tanmer-dev:~$ kubectl get po --all-namespaces -o wide
NAMESPACE     NAME                            READY     STATUS    RESTARTS   AGE       IP           NODE
kube-system   etcd-node1                      1/1       Running   0          1h        10.0.1.10    node1
kube-system   kube-apiserver-node1            1/1       Running   0          1h        10.0.1.10    node1
kube-system   kube-controller-manager-node1   1/1       Running   0          1h        10.0.1.10    node1
kube-system   kube-dns-86f4d74b45-lhf58       3/3       Running   2          1h        10.244.0.3   node1
kube-system   kube-flannel-ds-fwd78           1/1       Running   0          34m       10.0.1.11    node2
kube-system   kube-flannel-ds-hpf5s           1/1       Running   0          34m       10.0.1.10    node1
kube-system   kube-proxy-4vf42                1/1       Running   0          43m       10.0.1.11    node2
kube-system   kube-proxy-kvv94                1/1       Running   0          1h        10.0.1.10    node1
kube-system   kube-scheduler-node1            1/1       Running   0          1h        10.0.1.10    node1
```

## &#x20;安装Dashboard

```bash
kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/master/src/deploy/recommended/kubernetes-dashboard.yaml
```

这里，又需要翻墙下载镜像 `k8s.gcr.io/kubernetes-dashboard-amd64:v1.8.3`

> 官方文档：<https://github.com/kubernetes/dashboard>

Dashboard启动之后，通过下面命令，代理API端口

```bash
kubectl proxy
```

然后访问 <http://localhost:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/>

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LAclAvxu-naSsQ95Vf1%2F-LActE-Z1yMM0X01NeOt%2Fimage.png?alt=media\&token=75d1b23e-d520-4234-bace-62d87ebf65b2)

### 需要创建一个帐号

这里需要创建一个帐号才能访问。

```bash
cat <<EOS|kubectl apply -f -
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: admin-user
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: admin-user
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: admin-user
  namespace: kube-system
EOS

kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep admin-user | awk '{print $1}')
```

把输出的Token填入网站

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LAclAvxu-naSsQ95Vf1%2F-LAcuaKYgEkpH3HtOwiQ%2Fimage.png?alt=media\&token=54c5e998-3165-4b9e-8a04-974166f556bb)

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LAclAvxu-naSsQ95Vf1%2F-LAcueoDZ_fJYUCWdu9g%2Fimage.png?alt=media\&token=2959b30b-5d40-4b94-97e6-d62ed45fc0c4)

### 安装Heapster

> 官方文档：<https://github.com/kubernetes/heapster>

这里又是翻墙下载镜像：

```
k8s.gcr.io/heapster-influxdb-amd64:v1.3.3
k8s.gcr.io/heapster-amd64:v1.4.2
k8s.gcr.io/heapster-grafana-amd64:v4.4.3
```

```bash
# 安装influxdb
kubectl apply -f https://raw.githubusercontent.com/kubernetes/heapster/master/deploy/kube-config/influxdb/influxdb.yaml
# 安装heapster
kubectl apply -f https://raw.githubusercontent.com/kubernetes/heapster/master/deploy/kube-config/influxdb/heapster.yaml
# 安装grafana
kubectl apply -f https://raw.githubusercontent.com/kubernetes/heapster/master/deploy/kube-config/influxdb/grafana.yaml
# 分配权限
kubectl apply -f https://raw.githubusercontent.com/kubernetes/heapster/master/deploy/kube-config/rbac/heapster-rbac.yaml
```

这里我把k8s.gcr.io/heapster-influxdb-amd64:v1.3.3镜像到了tanmerk8s/heapster-influxdb-amd64:v1.3.3，所以下面这个方法更简单

```bash
curl https://raw.githubusercontent.com/kubernetes/heapster/master/deploy/kube-config/influxdb/influxdb.yaml | sed 's!image: k8s.gcr.io/!image: tanmerk8s/!' | kubectl apply -f -
curl https://raw.githubusercontent.com/kubernetes/heapster/master/deploy/kube-config/influxdb/heapster.yaml | sed 's!image: k8s.gcr.io/!image: tanmerk8s/!' | kubectl apply -f -
curl https://raw.githubusercontent.com/kubernetes/heapster/master/deploy/kube-config/influxdb/grafana.yaml | sed 's!image: k8s.gcr.io/!image: tanmerk8s/!' | kubectl apply -f -
```

这里Grafana的内部端口是`3000`，如果需要外网访问，可以修改一下deploy配置，设置密码保护，然后通过Ingress绑定域名实现外网访问。

现在能够看到，`Heapster`和`monitoring-grafana`已经安装成功

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LAd1frZXIiGo6lbiGid%2F-LAd2WKjpaHDQXKB0V_S%2Fimage.png?alt=media\&token=2b45193a-880f-4706-8681-1f4db44afc16)

Grafana service by default requests for a LoadBalancer. If that is not available in your cluster, consider changing that to NodePort. Use the external IP assigned to the Grafana service, to access Grafana. The default user name and password is 'admin'. Once you login to Grafana, add a datasource that is InfluxDB. The URL for InfluxDB will be `http://INFLUXDB_HOST:INFLUXDB_PORT`. Database name is 'k8s'. Default user name and password is 'root'.

Grafana访问地址：

```
http://localhost:8001/api/v1/namespaces/kube-system/services/monitoring-grafana/proxy
```

现在，Dashboard就能看到服务器的资源使用情况了

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LAd1frZXIiGo6lbiGid%2F-LAd4QYR88SgQJtgrZWe%2Fimage.png?alt=media\&token=58edeeee-c561-4c82-b746-fce7b452a2b9)

## 安装Weave Scope

Weave Scope是一个Kubernetes的可视化监控工具，通过他可以总览整个集群构架，实时诊断容器。

安装非常简单，一个命令搞定，最重要的是不用翻墙：

```bash
kubectl apply -f "https://cloud.weave.works/k8s/scope.yaml?k8s-version=$(kubectl version | base64 | tr -d '\n')"
```

```
xiaohui@node1:~$ kubectl -n weave get all
NAME                                   READY     STATUS    RESTARTS   AGE
pod/weave-scope-agent-lwxk9            1/1       Running   0          1m
pod/weave-scope-agent-mfdnc            1/1       Running   0          1m
pod/weave-scope-app-69f8c6745d-sfj75   1/1       Running   0          1m

NAME                      TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
service/weave-scope-app   ClusterIP   10.102.231.71   <none>        80/TCP    1m

NAME                               DESIRED   CURRENT   READY     UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
daemonset.apps/weave-scope-agent   2         2         2         2            2           <none>          1m

NAME                              DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/weave-scope-app   1         1         1            1           1m

NAME                                         DESIRED   CURRENT   READY     AGE
replicaset.apps/weave-scope-app-69f8c6745d   1         1         1         1m
```

映射Pod端口到本地`4040`端口

```bash
kubectl port-forward -n weave "$(kubectl get -n weave pod --selector=weave-scope-component=app -o jsonpath='{.items..metadata.name}')" 4040
```

本地浏览器访问 <http://localhost:4040>

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LAd6akokH5QLFpFeoXw%2F-LAd9MmCWKviFxW1Ffcq%2Fimage.png?alt=media\&token=e4559958-5d7d-41e1-acd9-e8efa434539f)

## 安装Ingress

要让服务能够被外部网络通过域名访问，我们需要安装Ingress。

官方文档：<https://kubernetes.io/docs/concepts/services-networking/ingress/>

这里我们可以使用官方的[`ingress-nginx`](https://github.com/kubernetes/ingress-nginx/blob/master/README.md)或[`traefik`](https://traefik.io/)，这里我们选用功能更完善的`traefik`

### Traefik

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LAdDnTjQ4JzgWFEg5jq%2F-LAdErNKJxdd1geHh-rb%2Fimage.png?alt=media\&token=f2533180-04d6-49aa-9a57-acb358f8d16b)

参考文档：

<https://github.com/rootsongjc/kubernetes-handbook/blob/master/practice/edge-node-configuration.md>

以下配置文件可以在[kubernetes-handbook](https://github.com/rootsongjc/kubernetes-handbook)GitHub仓库中的[../manifests/traefik-ingress/](https://github.com/rootsongjc/kubernetes-handbook/blob/master/manifests/traefik-ingress/)目录下找到。

#### 创建服务帐号

{% code title="" %}

```bash
cat <<EOS|kubectl apply -f -
apiVersion: v1
kind: ServiceAccount
metadata:
  name: ingress
  namespace: kube-system

---

kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: ingress
subjects:
  - kind: ServiceAccount
    name: ingress
    namespace: kube-system
roleRef:
  kind: ClusterRole
  name: cluster-admin
  apiGroup: rbac.authorization.k8s.io
EOS
```

{% endcode %}

#### 部署Traefik DaemonSet

{% code title="" %}

```bash
cat <<EOS|kubectl apply -f -
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  name: traefik-ingress-lb
  namespace: kube-system
  labels:
    k8s-app: traefik-ingress-lb
spec:
  template:
    metadata:
      labels:
        k8s-app: traefik-ingress-lb
        name: traefik-ingress-lb
    spec:
      terminationGracePeriodSeconds: 60
      hostNetwork: true
      restartPolicy: Always
      serviceAccountName: ingress
      containers:
      - image: traefik
        name: traefik-ingress-lb
        resources:
          limits:
            cpu: 200m
            memory: 30Mi
          requests:
            cpu: 100m
            memory: 20Mi
        ports:
        - name: http
          containerPort: 80
          hostPort: 80
        - name: admin
          containerPort: 8580
          hostPort: 8580
        args:
        - --web
        - --web.address=:8580
        - --kubernetes
      nodeSelector:
        edgenode: "true"
EOS
```

{% endcode %}

{% hint style="info" %}
注意：我们使用了nodeSelector选择边缘节点来调度traefik-ingress-lb运行在它上面，所有你需要使用：
{% endhint %}

```bash
kubectl label nodes node2 edgenode=true
```

可以看到，应用已经启动了

```bash
xiaohui@tanmer-dev:~$ kubectl -n kube-system get ds
NAME                 DESIRED   CURRENT   READY     UP-TO-DATE   AVAILABLE   NODE SELECTOR                   AGE
kube-flannel-ds      2         2         2         2            2           beta.kubernetes.io/arch=amd64   3h
kube-proxy           2         2         2         2            2           <none>                          3h
traefik-ingress-lb   1         1         1         1            1           edgenode=true                   1m
```

现在开启Web端管理服务，域名`traefik-ui.local`指向UI

{% code title="" %}

```bash
cat <<EOS|kubectl apply -f -
apiVersion: v1
kind: Service
metadata:
  name: traefik-web-ui
  namespace: kube-system
spec:
  selector:
    k8s-app: traefik-ingress-lb
  ports:
  - name: web
    port: 80
    targetPort: 8580
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: traefik-web-ui
  namespace: kube-system
spec:
  rules:
  - host: traefik-ui.local
    http:
      paths:
      - path: /
        backend:
          serviceName: traefik-web-ui
          servicePort: web
EOS
```

{% endcode %}

本地电脑改一下/etc/hosts文件，指向node2就能访问traefik web UI了

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LAdDnTjQ4JzgWFEg5jq%2F-LAdMuV_5iS_6QLHO9Xw%2Fimage.png?alt=media\&token=653dcf12-dfca-47ae-9993-154f2f86a28b)

## 翻墙下载镜像汇总

```bash
从国外下载镜像

images=(
k8s.gcr.io/pause-amd64:3.1
k8s.gcr.io/etcd-amd64:3.1.12
k8s.gcr.io/kube-apiserver-amd64:v1.10.1
k8s.gcr.io/kube-controller-manager-amd64:v1.10.1
k8s.gcr.io/kube-scheduler-amd64:v1.10.1
k8s.gcr.io/kube-proxy-amd64:v1.10.1
k8s.gcr.io/k8s-dns-sidecar-amd64:1.14.8
k8s.gcr.io/k8s-dns-dnsmasq-nanny-amd64:1.14.8
k8s.gcr.io/k8s-dns-kube-dns-amd64:1.14.8
k8s.gcr.io/kubernetes-dashboard-amd64:v1.8.3
k8s.gcr.io/heapster-influxdb-amd64:v1.3.3
k8s.gcr.io/heapster-amd64:v1.4.2
k8s.gcr.io/heapster-grafana-amd64:v4.4.3
k8s.gcr.io/kube-keepalived-vip:0.11
)

for sourceImageName in ${images[@]} ; do
    targetImageName=docker.corp.tanmer.com/tanmer/dockers/${sourceImageName}
    (    docker pull ${sourceImageName} \
      && docker tag ${sourceImageName} ${targetImageName} \
      && docker push ${targetImageName} \
      && docker rmi ${targetImageName} \
    ) || break
done


恢复镜像到国内

for targetImageName in ${images[@]} ; do
    sourceImageName=docker.corp.tanmer.com/tanmer/dockers/${targetImageName}
    (    docker pull ${sourceImageName} \
      && docker tag ${sourceImageName} ${targetImageName} \
      && docker rmi ${sourceImageName} \
    ) || break
done
```


# Rancher方式安装Kubernetes

Rancher 官方文档 <https://rancher.com/docs/rancher/v1.6/en/kubernetes/>

{% content-ref url="/pages/-LAI0Rbvy4BiDLR7A0fK" %}
[RBAC集成](/kubernetes/tong-guo-rancher-guan-li-kubernetes/rbac-ji-cheng)
{% endcontent-ref %}


# RBAC集成

官方文档 <https://rancher.com/docs/rancher/v1.6/en/kubernetes/rbac/>

```yaml
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: edit-dev
  namespace: dev # Specify which namespace you want these permissions granted in
subjects:
  - kind: User
    name: developer1
  - kind: User
    name: developer2
roleRef:
  kind: ClusterRole
  name: edit # Specify which type of role you want the users to have
  apiGroup: rbac.authorization.k8s.io
```

给Namespace `dev`添加编辑人员


# rke方式安装Kubernetes

上面我们知道了有两种方式安装集群：`kubeadm`和`rancher`，第一种防止如果是在国外服务器，是最简单的部署方式，但是到了国内，因为`gcr.io`被墙，就对部署造成了很大麻烦。第二种方式，Rancher 1.x目前只可以部署1.8.5版本的集群，无法体验更成熟的1.10版本，还有几个问题也是让我不喜欢用Rancher的原因：

1. 必须安装Rancher Server，浪费了一台服务器
2. Rancher的网络组件性能问题，应该没有Flannel快
3. 权限控制方面，要用kubectl控制集群，必须分配一个Rancher可编辑权限的帐号，这个帐号可以在Rancher的Web端进入任何一个容器，让Kubernetes的 RBAC 大打折扣。

因为Rancher 1.x的这些原因，Rancher推出了一个部署k8s原生集群的工具`rke`，他可以解决墙的问题，自动帮我们配置`etcd`集群，master集群，实现高可用。自动帮我们增加、删除节点，让集群管理变得非常简单。

下面我们开始正题

## 准备服务器节点

至少3台服务器，保证etcd数据高可用

### 配置ssh密钥登录

每台服务器配置ssh密钥登录，让当前运行rke程序的电脑能够无密码登录3台服务器

```bash
mkdir ~/.ssh
vi ~/.ssh/authorized_keys
```

把本地公钥`~/.ssh/id_rsa.pub`内容粘贴进3台服务器上的`~/.ssh/authorized_keys`文件

### 安装docker

这里用官方推荐的稳定版本，不要尝试最新版，避免后面出现奇怪问题

```bash
curl https://releases.rancher.com/install-docker/1.13.sh | sh
sudo usermod -aG docker ubuntu
cat <<JSON|sudo tee /etc/docker/daemon.json
{
  "registry-mirrors": ["https://registry.docker-cn.com"]
}
JSON
sudo systemctl restart docker
```

装完之后，设置国内镜像源

## 下载rke

rke 的github地址，<https://github.com/rancher/rke> 在这里，我们可以下载最新版本的rke二进制文件

## 配置集群

### 生成配置文件

下面命令，根据一路提示，即可完成配置，创建`cluster.yml`文件，这个文件一定要保存好，因为以后升级集群就靠他了。

```bash
rke config
```

这里有一点比较迷糊的得放得注意，`[+] Cluster Level SSH Private Key Path [~/.ssh/id_rsa]:`这个是说运行`rke`的电脑登录k8s节点的密钥文件地址，后面每个Host都会提示的`[+] SSH Private Key Path of host (192.180.10.1) [none]:`表示如果这台节点的登录密钥不是上面填的`Cluster Level`密钥，那就在这里指定另外一个文件，否者就按回车默认。

生成cluster.yml文件之后，手工添加以下配置：

```yaml
kubelet:
  image: rancher/hyperkube:v1.10.1-rancher1
  extra_args:
    read-only-port: 10255
ingress:
  provider: none
  options: {}
  node_selector: {}
  extra_args: {}
kubernetes_version: "1.10.1"
```

这个`read-only-port`可以让后面安装的`Heapster`能够获取到节点的硬件资源占用情况。

这里`ingress.provider`设置为`none`，是因为我们用`Traefik`作为Ingress服务，不用rke配置的默认`Nginx Ingress`

### 初始化集群

```bash
rke up
```

初始化成功之后，我可以用以下命令检查一下状态：

```
$ kubectl cluster-info
Kubernetes master is running at https://10.10.186.24:6443
KubeDNS is running at https://10.10.186.24:6443/api/v1/namespaces/kube-system/services/kube-dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
```

```
$ kubectl get no
NAME            STATUS    ROLES                      AGE       VERSION
10.10.149.122   Ready     controlplane,etcd,worker   33m       v1.10.1
10.10.17.220    Ready     controlplane,etcd,worker   33m       v1.10.1
10.10.186.24    Ready     controlplane,etcd,worker   33m       v1.10.1
```

### 安装Traefik

安装Traefik之后，我们就能够通过域名访问服务，也可以通过Traefik自动申请证书。

Consul和Traefik安装参考：

{% content-ref url="/pages/-LBo4VxO3EC\_dWi8Eil0" %}
[Traefik配置](/kubernetes/traefik-pei-zhi)
{% endcontent-ref %}

### 安装Keepalived for Kubernetes

<https://github.com/kubernetes/contrib/tree/master/keepalived-vip>

教程中，以下几个地方指定注意：

#### ConfigMap:

```
apiVersion: v1
kind: ConfigMap
metadata:
  name: vip-configmap
  namespace: kube-system
data:
  10.10.140.202: ''

```

默认的data配置是''10.4.0.50: default/echoheaders"，10.4.0.50是VIP，default是k8s的命名空间，echoheaders是k8s中default空间的服务，这个配置的意思是路由10.4.0.50到default空间的echoheaders服务。

但是，我们用到Keepalived的目的并不是他的路由功能，而是让VIP自动在Traefik所在的几台服务器上漂移，实现Traefik高可用，服务路由的事交给Traefik就好。

因此，我们把data的配置改成`10.10.140.202: ''`

#### DaemonSet：

```
    spec:
      hostNetwork: true
      serviceAccount: kube-keepalived-vip
```

因为我们启用了RBAC，所以要加上serviceAccount，怎么创建和配置，看<https://github.com/kubernetes/contrib/tree/master/keepalived-vip#optional-install-the-rbac-policies>

```
          args:
          - --services-configmap=kube-system/vip-configmap
          # unicast uses the ip of the nodes instead of multicast
          # this is useful if running in cloud providers (like AWS)
          - --use-unicast=true
          # vrrp version can be set to 2.  Default 3.
          #- --vrrp-version=2
```

这里修改参数`--services-configmap=kube-system/vip-configmap`，因为我们把configmap放到了`kube-system`空间，`--use-unicast=true`是为了解决云服务商屏蔽了多播(multicast)数据。

当运行3个Keepalived节点时，第3个节点出现错误提示，原因是`vrrp version 3`会报错，`vrrp version 2`没有问题。但是当前的docker image是`k8s.gcr.io/kube-keepalived-vip:0.11`，不是最新的版本，文档里说了支持参数`--vrrp-version`，可镜像是老的不支持。自己编译镜像比较麻烦，主要是翻墙问题和Go语言环境配置。这里有个简单的方法就是添加一个entrypoint.sh文件，启动时执行这个文件，把version替换成2

```
apiVersion: v1
kind: ConfigMap
metadata:
  name: vip-entrypoint
  namespace: kube-system
  labels:
    app: keepalived
data:
  entrypoint.sh: |
    sed -i 's/vrrp_version 3/vrrp_version 2/' /keepalived.tmpl && \
    /kube-keepalived-vip \
    --services-configmap=kube-system/vip-configmap \
    --use-unicast=true

```

```
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  name: kube-keepalived-vip
  namespace: kube-system
  labels:
    app: keepalived
spec:
  template:
    metadata:
      labels:
        name: kube-keepalived-vip
        app: keepalived
    spec:
      hostNetwork: true
      serviceAccount: kube-keepalived-vip
      containers:
        - image: tanmerk8s/kube-keepalived-vip:0.11
          name: kube-keepalived-vip
          imagePullPolicy: IfNotPresent
          securityContext:
            privileged: true
          volumeMounts:
            - mountPath: /lib/modules
              name: modules
              readOnly: true
            - mountPath: /dev
              name: dev
            - mountPath: /mybin
              name: entrypoint
          # use downward API
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          command:
          - bash
          - /mybin/entrypoint.sh
          # to use unicast
          # args:
          # - --services-configmap=kube-system/vip-configmap
          # # unicast uses the ip of the nodes instead of multicast
          # # this is useful if running in cloud providers (like AWS)
          # - --use-unicast=true
          # # vrrp version can be set to 2.  Default 3.
          # - --vrrp-version=2
      volumes:
        - name: modules
          hostPath:
            path: /lib/modules
        - name: dev
          hostPath:
            path: /dev
        - name: entrypoint
          configMap:
            name: vip-entrypoint
            items:
              - key: entrypoint.sh
                path: entrypoint.sh
      nodeSelector:
        edgenode: "true"

```


# RBAC用户管理

## 方式认证

### X509 Client Certs

#### 创建用户

参考文档：&#x20;

* <https://docs.bitnami.com/kubernetes/how-to/configure-rbac-in-your-kubernetes-cluster/>
* <https://kubernetes.io/docs/admin/authorization/rbac/#role-examples>
*

**第一步：生成用户证书**

在k8s master节点执行：

```bash
sudo -s
mkdir -p ~/k8s-user-certs
cd ~/k8s-user-certs
cert_user=wenlg
cert_group=deployer
openssl genrsa -out ${cert_user}.key 2048
openssl req -new -key ${cert_user}.key -out ${cert_user}.csr -subj "/CN=${cert_user}/O=${cert_group}"
openssl x509 -req -in ${cert_user}.csr -CA /etc/kubernetes/pki/ca.crt -CAkey /etc/kubernetes/pki/ca.key -CAcreateserial -out ${cert_user}.crt -days 365
```

下载生成的证书`wenlg.crt`和密钥`wenlg.key`，保存在自己电脑上，推荐存放在`~/.certs/`目录

在本地电脑执行：

```bash
cert_user=wenlg
cert_namespace=office
kubectl config set-credentials ${cert_user} --client-certificate=$(realpath ~/.certs/${cert_user}.crt)  --client-key=$(realpath ~/.certs/${cert_user}.key) --embed-certs=true
kubectl config set-context ${cert_user}@kubernetes --cluster=kubernetes --user=${cert_user} --namespace=${cert_namespace}
```

本地执行，如果提示`Error from server (Forbidden): pods is forbidden: User "wenlg" cannot list pods in the namespace "default"` 就说明证书可以了。

```bash
kubectl --context=${cert_user}@kubernetes get pods
```

**配置权限**

创建一个角色`deployment-manager`：

```bash
kubectl create namespace ${cert_namespace}
cat <<EOS|kubectl apply -f -
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  namespace: ${cert_namespace}
  name: deployment-manager
rules:
- apiGroups: ["", "extensions", "apps"]
  resources: ["deployments", "replicasets", "pods"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
EOS
```

绑定角色和用户：

```bash
cat <<EOS|kubectl apply -f -
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: deployment-manager-binding
  namespace: ${cert_namespace}
subjects:
- kind: User
  name: ${cert_user}
  apiGroup: ""
roleRef:
  kind: Role
  name: deployment-manager
  apiGroup: ""
EOS
```

&#x20;测试一下权限：

```bash
# 部署一个镜像
kubectl --context=${cert_user}@kubernetes run --image citizenstig/httpbin httpbin
# 显示部署和pod列表
kubectl --context=${cert_user}@kubernetes get deploy,po
```

{% code title="Output" %}

```
➜  ~ kubectl --context=${cert_user}@kubernetes get deploy,po
NAME             DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deploy/httpbin   1         1         1            1           47s

NAME                          READY     STATUS    RESTARTS   AGE
po/httpbin-5c5449d8b8-qr2dz   1/1       Running   0          47s
```

{% endcode %}

尝试访问`default` namespace的资源

```bash
kubectl --context=${cert_user}@kubernetes get po --namespace=default
```

{% code title="Output" %}

```
➜  ~ kubectl --context=${cert_user}@kubernetes get po --namespace=default
Error from server (Forbidden): pods is forbidden: User "wenlg" cannot list pods in the namespace "default"
```

{% endcode %}

至此，创建一个用户，设定访问权限就完成了。

### Webhook Token

参考文档：<https://kubernetes.io/docs/admin/authentication/#webhook-token-authentication>


# Traefik配置

官方文档：<https://github.com/containous/traefik/blob/master/docs/configuration/backends/kubernetes.md>

## 安装Consul

因为我们要配置Traefik高可用，数据存储就得用到Consul KV Store。

给3台服务器打上标签，`Consul`会以`StatefulSet`模式安装在上面

```bash
kubectl label nodes node1 node2 node2 consul=server
```

创建服务

```bash
cat <<YAML | kubectl apply -f -
apiVersion: v1
kind: Service
metadata:
  name: consul
  namespace: kube-system
  labels:
    app: consul
spec:
  clusterIP: None
  ports:
    - name: http
      port: 8500
      targetPort: 8500
    - name: https
      port: 8443
      targetPort: 8443
    - name: rpc
      port: 8400
      targetPort: 8400
    - name: serflan-tcp
      protocol: "TCP"
      port: 8301
      targetPort: 8301
    - name: serflan-udp
      protocol: "UDP"
      port: 8301
      targetPort: 8301
    - name: serfwan-tcp
      protocol: "TCP"
      port: 8302
      targetPort: 8302
    - name: serfwan-udp
      protocol: "UDP"
      port: 8302
      targetPort: 8302
    - name: server
      port: 8300
      targetPort: 8300
    - name: consuldns
      port: 8600
      targetPort: 8600
  selector:
    app: consul
YAML
```

创建`StatefulSet`，数据存储在节点的`/mnt/consul`目录

```bash
cat <<YAML | kubectl apply -f -
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
  name: consul
  namespace: kube-system
spec:
  serviceName: consul
  replicas: 3
  template:
    metadata:
      labels:
        app: consul
    spec:
      nodeSelector:
        consul: server
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchExpressions:
                  - key: consul
                    operator: In
                    values:
                      - server
              topologyKey: kubernetes.io/hostname
      terminationGracePeriodSeconds: 10
      securityContext:
        fsGroup: 1000
      containers:
      - name: consul
        image: consul:0.9.2
        args:
        - "agent"
        - "-advertise=\$(POD_IP)"
        - "-bind=0.0.0.0"
        - "-bootstrap-expect=3"
        - "-retry-join=consul-0.consul.\$(NAMESPACE).svc.cluster.local"
        - "-retry-join=consul-1.consul.\$(NAMESPACE).svc.cluster.local"
        - "-retry-join=consul-2.consul.\$(NAMESPACE).svc.cluster.local"
        - "-client=0.0.0.0"
        - "-datacenter=fr"
        - "-data-dir=/consul/data"
        - "-domain=cluster.local"
        - "-server"
        - "-ui"
        - "-disable-host-node-id"
        ports:
        - containerPort: 8500
          name: ui-port
        - containerPort: 8400
          name: alt-port
        - containerPort: 53
          name: udp-port
        - containerPort: 8443
          name: https-port
        - containerPort: 8080
          name: http-port
        - containerPort: 8301
          name: serflan
        - containerPort: 8302
          name: serfwan
        - containerPort: 8600
          name: consuldns
        - containerPort: 8300
          name: server
        env:
        - name: POD_IP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        - name: NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        lifecycle:
          preStop:
            exec:
              command:
              - /bin/sh
              - -c
              - consul leave
        volumeMounts:
        - name: ca-certificates
          mountPath: /etc/ssl/certs
        - name: consul-data
          mountPath: /consul/data
      volumes:
      - name: ca-certificates
        hostPath:
          path: /usr/share/ca-certificates/
      - name: consul-data
        hostPath:
          path: /mnt/consul
YAML
```

## 部署traefik

执行下面代码，即可以部署`traefix`，执行前需要修改Email `me@mydomain` 为你自己的，注意，很多教程都没有提到这个Email有什么讲究，其实只要填写你自己真实的Email就可以了，Traefik会自动用你的邮箱去Let's Encrpyt注册账号，自动通过API获取域名验证字符串，Let's Encrpyt验证域名时，Traefik会自动识别URL <http://your.domain/.wellknown/xxxx/xxxx> (我忘了具体URL是什么)，返回验证字符串，实现域名身份验证。这一切都是制动的，你需要做的就是提前把域名解析到Traefik的外网IP，提供一个真实的Email。

3个节点打上标签`edgenode=true`，Traefik会部署在满足这个标签的服务器上:

```bash
kubectl label nodes node1 node2 node3 edgenode=true
```

开始部署：

```bash
cat <<YAML | kubectl apply -f -
apiVersion: v1
kind: ServiceAccount
metadata:
  name: ingress
  namespace: kube-system
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: ingress
subjects:
  - kind: ServiceAccount
    name: ingress
    namespace: kube-system
roleRef:
  kind: ClusterRole
  name: cluster-admin
  apiGroup: rbac.authorization.k8s.io

---
kind: ConfigMap
apiVersion: v1
data:
  traefik.toml: |
    checkNewVersion = false
    logLevel = "INFO"
    #defaultEntryPoints = ["http", "https"]
    defaultEntryPoints = ["http"]

    [retry]
    attempts = 3

    [entryPoints]
      [entryPoints.http]
      address = ":80"
      #  [entryPoints.http.redirect]
      #  entryPoint = "https"
      [entryPoints.https]
      address = ":443"
        [entryPoints.https.tls]

    [consul]
    endpoint = "consul.kube-system:8500"
    watch = true
    prefix = "traefik"

    [acme]
    email = "me@mydomain"
    storage = "traefik/acme/account"
    entryPoint = "https"
    OnHostRule = true
    acmeLogging = true
    #caServer = "https://acme-staging-v02.api.letsencrypt.org/directory"
    [acme.httpChallenge]
    entryPoint = "http"

metadata:
  name: traefik-conf
  namespace: kube-system

---
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  name: traefik-ingress-lb
  namespace: kube-system
  labels:
    k8s-app: traefik-ingress-lb
spec:
  template:
    metadata:
      labels:
        k8s-app: traefik-ingress-lb
        name: traefik-ingress-lb
    spec:
      terminationGracePeriodSeconds: 60
      hostNetwork: true
      dnsPolicy: ClusterFirstWithHostNet
      restartPolicy: Always
      serviceAccountName: ingress
      volumes:
      - name: config
        configMap:
          name: traefik-conf
      containers:
      - image: traefik
        name: traefik-ingress-lb
        resources:
          limits:
            cpu: 200m
            memory: 30Mi
          requests:
            cpu: 100m
            memory: 20Mi
        ports:
        - name: http
          containerPort: 80
          hostPort: 80
        - name: https
          containerPort: 443
          hostPort: 443
        - name: admin
          containerPort: 8580
          hostPort: 8580
        volumeMounts:
        - mountPath: "/config"
          name: config
        args:
        - --web
        - --web.address=:8580
        - --kubernetes
        - --configfile=/config/traefik.toml
      nodeSelector:
        edgenode: "true"
---
apiVersion: v1
kind: Service
metadata:
  name: traefik-web-ui
  namespace: kube-system
spec:
  selector:
    k8s-app: traefik-ingress-lb
  ports:
  - name: web
    port: 80
    targetPort: 8580
YAML
```

这个配置文件，是不会自动`http`重定向到`https`的，需要在独立的`ingress`中去声明`annotations`

支持的`annotations` 列表：<https://github.com/containous/traefik/blob/master/docs/configuration/backends/kubernetes.md#general-annotations>

重定向`http`到`https`: `traefik.ingress.kubernetes.io/redirect-entry-point: https`

查看部署状态：

```
$ kubectl -n kube-system get po -l k8s-app=traefik-ingress-lb
NAME                       READY     STATUS    RESTARTS   AGE
traefik-ingress-lb-27bql   1/1       Running   0          2m
traefik-ingress-lb-78tpl   1/1       Running   0          2m
traefik-ingress-lb-xqncg   1/1       Running   0          2m
```

查看Web界面 <http://127.0.0.1:8580>

```bash
kubectl -n kube-system port-forward traefik-ingress-lb-27bql 8580
```

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LDwe5Z6jc6Bv8TLPfjK%2F-LDwilRctmymM9PtKAlo%2Fimage.png?alt=media\&token=bfd609d2-6cea-455c-a2f1-de8cd2b0cc80)

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LDwe5Z6jc6Bv8TLPfjK%2F-LDwiofZn03r4R0ZfKVm%2Fimage.png?alt=media\&token=7ff86a1c-260e-4cb2-8c19-835a3d763555)

## 注册Let'sencrypt账号


# 创建etcd集群

官方文档：<https://kubernetes.io/docs/setup/independent/high-availability/#before-you-begin>

Kubernetes的数据库是`etcd`，要让k8s实现高可用，首先就需要实现etcd的高可用，因此本文介绍如何创建etcd集群

`etcd`集群需要奇数个节点才能选举出`leader`，因此我们至少需要`3`个节点来运行`etcd`

## 安装etcd

因为我们初始化k8s集群时，第一个`master`已经安装来`etcd`，比如这个master节点名叫`node1`，所以我们还需要两个节点：`node2`, `node3`

安装etcd之前，我们需要先在`node1`上检查etcd的版本：

```bash
root@10-9-126-15:~# grep image /etc/kubernetes/manifests/etcd.yaml
    image: k8s.gcr.io/etcd-amd64:3.1.12
```

确定版本为`3.1.12`

进入`etcd`下载页面：<https://github.com/coreos/etcd/releases>，根据Linux安装脚本，开始在`node2`和`node3`上面安装

把每个节点的IP地址记录在变量中，后面会用到：

```
node1_ip=10.9.126.15
node2_ip=10.9.65.99
node3_ip=10.9.28.242
```

```bash
ETCD_VER=v3.1.12
GITHUB_URL=https://github.com/coreos/etcd/releases/download
DOWNLOAD_URL=${GITHUB_URL}

rm -f /tmp/etcd-${ETCD_VER}-linux-amd64.tar.gz
rm -rf /tmp/etcd-download-test && mkdir -p /tmp/etcd-download-test

curl -L ${DOWNLOAD_URL}/${ETCD_VER}/etcd-${ETCD_VER}-linux-amd64.tar.gz -o /tmp/etcd-${ETCD_VER}-linux-amd64.tar.gz
tar xzvf /tmp/etcd-${ETCD_VER}-linux-amd64.tar.gz -C /tmp/etcd-download-test --strip-components=1
rm -f /tmp/etcd-${ETCD_VER}-linux-amd64.tar.gz

/tmp/etcd-download-test/etcd --version
<<COMMENT
etcd Version: 3.1.12
Git SHA: 918698add
Go Version: go1.8.7
Go OS/Arch: linux/amd64
COMMENT

ETCDCTL_API=3 /tmp/etcd-download-test/etcdctl version
<<COMMENT
etcdctl version: 3.1.12
API version: 3.1
COMMENT
```

为了简化安装流程，我们先用`apt`安装`etcd`，然后把我们指定版本的etcd二进制文件覆盖apt安装的文件：

```bash
apt-get update
apt-get install etcd -y
```

把之前下载额二进制文件覆盖apt安装的文件：

```bash
systemctl stop etcd
mv /tmp/etcd-download-test/etcd /usr/bin/
mv /tmp/etcd-download-test/etcdctl /usr/bin/
systemctl start etcd
systemctl status etcd
```

在`node2`和`node3`上修改配置文件：

未完。。。


# Kubeapps

`Kubeapps` 是集`CLI`和`GUI`一体的`Helm` charts部署工具。比如一个复杂的项目，有API端，后端，前端，Worker端，每个端都是需要独立部署，他们都有各自的chart，要一次性部署好这几个端，通过`kubeapps`一个命令就可以搞定。

## 下载

官方文档：<https://github.com/kubeapps/kubeapps/blob/master/docs/getting-started.md#installation-of-the-kubeapps-installer>

在`github`的[`release`](https://github.com/kubeapps/kubeapps/releases)页面，下载最新版，放到bin目录里

```bash
mv ~/Downloads/kubeapps-darwin-amd64 /usr/local/bin/kubeapps
chmod +x /usr/local/bin/kubeapps
```

## 安装

```bash
kubeapps up
```

{% code title="Output" %}

```
➜  tamigos kubeapps up
INFO[0000] Fetching schemas for 44 resources
INFO[0002] Updating customresourcedefinitions apprepositories.kubeapps.com
INFO[0002]  Creating non-existent customresourcedefinitions apprepositories.kubeapps.com
INFO[0003] Updating clusterroles apprepository-controller
INFO[0003]  Creating non-existent clusterroles apprepository-controller
INFO[0003] Updating clusterrolebindings apprepository-controller
INFO[0003]  Creating non-existent clusterrolebindings apprepository-controller
INFO[0003] Updating customresourcedefinitions cronjobtriggers.kubeless.io
INFO[0003]  Creating non-existent customresourcedefinitions cronjobtriggers.kubeless.io
INFO[0003] Updating customresourcedefinitions functions.kubeless.io
INFO[0003]  Creating non-existent customresourcedefinitions functions.kubeless.io
INFO[0003] Updating customresourcedefinitions httptriggers.kubeless.io
INFO[0003]  Creating non-existent customresourcedefinitions httptriggers.kubeless.io
INFO[0003] Updating namespaces kubeapps
INFO[0003]  Creating non-existent namespaces kubeapps
INFO[0003] Updating namespaces kubeless
INFO[0003]  Creating non-existent namespaces kubeless
INFO[0003] Updating clusterroles kubeless-controller-deployer
INFO[0003]  Creating non-existent clusterroles kubeless-controller-deployer
INFO[0003] Updating clusterrolebindings kubeless-controller-deployer
INFO[0004]  Creating non-existent clusterrolebindings kubeless-controller-deployer
INFO[0004] Updating clusterrolebindings sealed-secrets-controller
INFO[0004]  Creating non-existent clusterrolebindings sealed-secrets-controller
INFO[0004] Updating customresourcedefinitions sealedsecrets.bitnami.com
INFO[0004]  Creating non-existent customresourcedefinitions sealedsecrets.bitnami.com
INFO[0004] Updating clusterroles secrets-unsealer
INFO[0004]  Creating non-existent clusterroles secrets-unsealer
INFO[0004] Updating clusterrolebindings tiller-cluster-admin
INFO[0004]  Creating non-existent clusterrolebindings tiller-cluster-admin
INFO[0004] Updating customresourcedefinitions kubeapps.helmreleases.helm.bitnami.com
INFO[0004]  Creating non-existent customresourcedefinitions kubeapps.helmreleases.helm.bitnami.com
INFO[0004] Updating rolebindings kube-system.sealed-secrets-controller
INFO[0004]  Creating non-existent rolebindings kube-system.sealed-secrets-controller
INFO[0004] Updating services kube-system.sealed-secrets-controller
INFO[0004]  Creating non-existent services kube-system.sealed-secrets-controller
INFO[0004] Updating serviceaccounts kube-system.sealed-secrets-controller
INFO[0005]  Creating non-existent serviceaccounts kube-system.sealed-secrets-controller
INFO[0005] Updating roles kube-system.sealed-secrets-key-admin
INFO[0005]  Creating non-existent roles kube-system.sealed-secrets-key-admin
INFO[0005] Updating roles kubeapps.apprepository-controller
INFO[0005]  Creating non-existent roles kubeapps.apprepository-controller
INFO[0005] Updating rolebindings kubeapps.apprepository-controller
INFO[0005]  Creating non-existent rolebindings kubeapps.apprepository-controller
INFO[0005] Updating serviceaccounts kubeapps.apprepository-controller
INFO[0005]  Creating non-existent serviceaccounts kubeapps.apprepository-controller
INFO[0005] Updating services kubeapps.chartsvc
INFO[0005]  Creating non-existent services kubeapps.chartsvc
INFO[0005] Updating apprepositories kubeapps.incubator
INFO[0005]  Creating non-existent apprepositories kubeapps.incubator
INFO[0005] Updating services kubeapps.kubeapps
INFO[0006]  Creating non-existent services kubeapps.kubeapps
INFO[0006] Updating services kubeapps.kubeapps-dashboard-ui
INFO[0006]  Creating non-existent services kubeapps.kubeapps-dashboard-ui
INFO[0006] Updating configmaps kubeapps.kubeapps-dashboard-ui-vhost-425de41
INFO[0006]  Creating non-existent configmaps kubeapps.kubeapps-dashboard-ui-vhost-425de41
INFO[0006] Updating configmaps kubeapps.kubeapps-vhost-5811f90
INFO[0006]  Creating non-existent configmaps kubeapps.kubeapps-vhost-5811f90
INFO[0006] Updating secrets kubeapps.mongodb
INFO[0006]  Creating non-existent secrets kubeapps.mongodb
INFO[0006] Updating services kubeapps.mongodb
INFO[0006]  Creating non-existent services kubeapps.mongodb
INFO[0006] Updating persistentvolumeclaims kubeapps.mongodb-data
INFO[0006]  Creating non-existent persistentvolumeclaims kubeapps.mongodb-data
INFO[0007] Updating apprepositories kubeapps.stable
INFO[0007]  Creating non-existent apprepositories kubeapps.stable
INFO[0007] Updating apprepositories kubeapps.svc-cat
INFO[0007]  Creating non-existent apprepositories kubeapps.svc-cat
INFO[0007] Updating serviceaccounts kubeapps.tiller
INFO[0007]  Creating non-existent serviceaccounts kubeapps.tiller
INFO[0007] Updating serviceaccounts kubeless.controller-acct
INFO[0007]  Creating non-existent serviceaccounts kubeless.controller-acct
INFO[0007] Updating configmaps kubeless.kubeless-config
INFO[0008]  Creating non-existent configmaps kubeless.kubeless-config
INFO[0008] Updating deployments kube-system.sealed-secrets-controller
INFO[0008]  Creating non-existent deployments kube-system.sealed-secrets-controller
INFO[0008] Updating deployments kubeapps.apprepository-controller
INFO[0008]  Creating non-existent deployments kubeapps.apprepository-controller
INFO[0008] Updating deployments kubeapps.chartsvc
INFO[0008]  Creating non-existent deployments kubeapps.chartsvc
INFO[0008] Updating deployments kubeapps.kubeapps
INFO[0008]  Creating non-existent deployments kubeapps.kubeapps
INFO[0008] Updating deployments kubeapps.kubeapps-dashboard-ui
INFO[0008]  Creating non-existent deployments kubeapps.kubeapps-dashboard-ui
INFO[0008] Updating deployments kubeapps.mongodb
INFO[0009]  Creating non-existent deployments kubeapps.mongodb
INFO[0009] Updating deployments kubeapps.tiller-deploy
INFO[0009]  Creating non-existent deployments kubeapps.tiller-deploy
INFO[0009] Updating deployments kubeless.kubeless-controller-manager
INFO[0009]  Creating non-existent deployments kubeless.kubeless-controller-manager

Kubeapps has been deployed successfully.
It may take a few minutes for all components to be ready.

NAMESPACE  	NAME                         	CLUSTER-IP   	EXTERNAL-IP	PORT(S)
kubeapps   	svc/chartsvc                 	10.43.208.41 	           	8080/TCP,
kubeapps   	svc/kubeapps                 	10.43.197.11 	           	8080/TCP,
kubeapps   	svc/kubeapps-dashboard-ui    	10.43.229.120	           	8080/TCP,
kubeapps   	svc/mongodb                  	10.43.136.139	           	27017/TCP,
kube-system	svc/sealed-secrets-controller	10.43.208.151	           	8080/TCP,

NAMESPACE  	NAME                              	DESIRED	CURRENT	UP-TO-DATE	AVAILABLE
kubeapps   	deploy/apprepository-controller   	1      	1      	1         	0
kubeapps   	deploy/chartsvc                   	1      	1      	1         	0
kubeapps   	deploy/kubeapps                   	1      	1      	1         	0
kubeapps   	deploy/kubeapps-dashboard-ui      	1      	1      	1         	0
kubeapps   	deploy/mongodb                    	1      	1      	1         	0
kubeapps   	deploy/tiller-deploy              	1      	1      	1         	0
kubeless   	deploy/kubeless-controller-manager	1      	1      	1         	0
kube-system	deploy/sealed-secrets-controller  	1      	1      	1         	0

NAMESPACE	NAME	DESIRED	CURRENT

NAMESPACE	NAME                                         	STATUS
kubeapps 	pod/apprepository-controller-7f4dc847df-jflhb	Pending
kubeapps 	pod/chartsvc-64f494f5f7-dzs49                	Pending
kubeapps 	pod/kubeapps-594c9d4fb9-w6skb                	Pending
kubeapps 	pod/kubeapps-dashboard-ui-9bb44d58b-tf2bn    	Pending
kubeapps 	pod/mongodb-55b55565ff-9tvj5                 	Pending

You can run `kubectl get all --all-namespaces -l created-by=kubeapps` to check the status of the Kubeapps components.
```

{% endcode %}

庆幸的是，kubeapps用的docker镜像不需要翻墙，有点不适应。

检查


# 工具

### Kube-shell

<https://github.com/cloudnativelabs/kube-shell>

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LAHuph7IkVqpyoBneaB%2F-LAHvZTJDCIZZQH2MmiK%2Fimage.png?alt=media\&token=2ac4521a-39ed-4ee6-8aec-f657d455f402)


# 安装Helm

## 初始化Tiller

`Helm`默认会把`Tiller`安装到`kube-system`空间，我不建议安装在这里，因为后面使用`helm`的用户必须得有`kube-system`空间`pod`的`list`权限，把最核心的pod暴露出来，谁也不愿意。因此，这里建议创建`tiller`空间

```bash
#!/bin/bash

# https://github.com/kubernetes/helm/issues/3460#issuecomment-385992094

set -e
tiller_namespace=tiller # default is kube-system
kubectl describe ns ${tiller_namespace} || kubectl create ns ${tiller_namespace}
kubectl create serviceaccount --namespace ${tiller_namespace} tiller
kubectl create clusterrolebinding tiller-cluster-rule --clusterrole=cluster-admin --serviceaccount=${tiller_namespace}:tiller
helm init --service-account tiller --tiller-image tanmerk8s/tiller:v2.9.0 --upgrade --tiller-namespace ${tiller_namespace}
```

## 配置用户权限

这里`cert_namespace`是需要授权的空间，`tiller_namespace`是我们上面初始化的`Tiller`所在的空间，下面例子给用户`gitlab`添加了对空间`project-staging`的`helm`部署权限。

```bash
#!/bin/bash
set -e

cert_namespace=project-staging
tiller_namespace=tiller

cat <<EOS|kubectl apply -f -
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: tiller-deployers
  namespace: ${tiller_namespace}
rules:
- apiGroups: ["*"]
  resources: ["pods"]
  verbs: ["list"]
- apiGroups: ["*"]
  resources: ["pods/portforward"]
  verbs: ["create"]

---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: deployers
  namespace: ${cert_namespace}
rules:
- apiGroups: ["*"]
  resources: ["*"]
  verbs: ["*"]
EOS

cat <<EOS|kubectl apply -f -
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: deployment
  namespace: ${tiller_namespace}
subjects:
- kind: User
  name: gitlab
  apiGroup: ""
roleRef:
  kind: Role
  name: tiller-deployers
  apiGroup: ""
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: deployment
  namespace: ${cert_namespace}
subjects:
- kind: User
  name: gitlab
  apiGroup: ""
roleRef:
  kind: Role
  name: deployers
  apiGroup: ""
EOS
```


# 亲和度配置

## Node亲和度

### 限制Pod调度到Node

```bash
kubectl taint node 10.100.0.133 backend=true:NoSchedule
```

这个命令表示在`node` `10.100.0.133`上，只有`pod`打上标签`backend=true`才能被调度到这个节点。

## Pod亲和度


# 文件系统


# GlusterFS

k8s集群中，怎么也会遇到数据持久化的问题，需要用到PV和PVC，把数据存储到一个固定的节点上。

[`GlusterFS`](https://docs.gluster.org/en/latest/Administrator%20Guide/GlusterFS%20Introduction/) 是一个可伸缩的分布式文件存储方案，Kubernetes 的StorageClass支持的驱动中，推荐  使用GlusterFS

<https://kubernetes.io/docs/concepts/storage/storage-classes/>

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LBMCpuRMtdrMmnBkK-A%2F-LBMET5iB4PmAiUZvZEL%2Fimage.png?alt=media\&token=fc8515f4-1c08-4123-a378-745a1bcee541)

## 开始安装&#x20;

GlusterFS官方文档：<https://docs.gluster.org/en/latest/Quick-Start-Guide/Quickstart/>

GlusterFS+Kubernetes官方文档：<https://github.com/gluster/gluster-kubernetes#quickstart>

因为文件系统功能简单，使用相对稳定，一次部署，基本上不会再做删减，因此，我们没有必要用Kubernetes去部署GlusterFS，手工部署反而更简单，依赖少。

### 安装前的准备

最少两台服务器，用replication模式，一主一备，防止数据库丢失，高可用。每台服务器2CPU，2G内存，100G SSD硬盘，网络带宽1G以上，安装Ubuntu 16.04

### 安装

```bash
sudo apt-get install software-properties-common
sudo apt-get update
sudo apt-get install glusterfs-server -y
```

## 配置

我们有两台服务器：`10.103.1.11`和`10.103.1.119`，在`10.103.1.11`上执行命令，连接`10.103.1.119`，组成集群

```bash
sudo gluster peer probe 10.103.1.119
sudo gluster peer status

Number of Peers: 1

Hostname: 10.103.1.119
Uuid: 60fedfe2-cb44-41f7-9b3d-cf2901793a85
State: Peer in Cluster (Connected)
```

格式化两台服务器的数据盘，自动挂载

```bash
mkfs.xfs -i size=512 -f /dev/vdb
# 设置开机自动挂载磁盘
echo "/dev/vdb /export/vdb xfs defaults 0 0"  >> /etc/fstab
# 创建目录
mkdir -p /export/vdb && mount -a && mkdir -p /export/vdb/brick1
```

### 添加第一个卷

在`10.103.1.11`上执行

```bash
gluster volume create gv0 replica 2 10.103.1.119:/export/vdb/brick1 10.103.1.11:/export/vdb/brick1
```

`gluster volume info`查看券状态

```
root@10-103-1-11:~# gluster volume info

Volume Name: gv0
Type: Replicate
Volume ID: 5faab39a-c2d4-47b8-a612-29f6d53616cf
Status: Created
Snapshot Count: 0
Number of Bricks: 1 x 2 = 2
Transport-type: tcp
Bricks:
Brick1: 10.103.1.119:/export/vdb/brick1
Brick2: 10.103.1.11:/export/vdb/brick1
Options Reconfigured:
transport.address-family: inet
performance.readdir-ahead: on
nfs.disable: on
```

启动卷：

```
root@10-103-1-11:~# gluster volume start gv0
volume start: gv0: success
root@10-103-1-11:~# gluster volume info

Volume Name: gv0
Type: Replicate
Volume ID: 5faab39a-c2d4-47b8-a612-29f6d53616cf
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 2 = 2
Transport-type: tcp
Bricks:
Brick1: 10.103.1.119:/export/vdb/brick1
Brick2: 10.103.1.11:/export/vdb/brick1
Options Reconfigured:
transport.address-family: inet
performance.readdir-ahead: on
nfs.disable: on
```

### 停止卷

```bash
root@10-103-1-11:~# gluster
gluster> volume stop kube-vol
Stopping volume will make its data inaccessible. Do you want to continue? (y/n) y
volume stop: kube-vol: success
```

### 删除卷

```
gluster> volume delete kube-vol
Deleting volume will erase all information about the volume. Do you want to continue? (y/n) y
volume delete: kube-vol: success
```

## 客户端挂载卷

&#x20;卷可以像普通nfs一样通过mount挂载，只需要安装glusterfs支持即可。

这里需要一天新的客户端电脑，安装`glusterfs`客户端：

```bash
sudo apt-get install software-properties-common
sudo add-apt-repository ppa:gluster/glusterfs-3.8
sudo apt-get update
sudo apt-get install glusterfs-client
```

### 挂载卷

```bash
sudo mkdir /mnt/my-vol
sudo mount -t glusterfs 10.103.1.11:/gv0/dir1 /mnt/my-vol
df -h /mnt/my-vol

root@10-10-61-207:~# df -h /mnt/my-vol
Filesystem        Size  Used Avail Use% Mounted on
10.103.1.11:/gv0  100G   33M  100G   1% /mnt/my-vol
```

## 开启磁盘配额

在服务端开启配额

```bash
gluster volume quota gv0 enable
```

在服务端设置一个目录的磁盘配额

```bash
gluster volume quota gv0 limit-usage / 10GB
```

客户端查看配额

```bash
root@10-10-61-207:~# df -h /mnt/my-vol
Filesystem        Size  Used Avail Use% Mounted on
10.103.1.11:/gv0   20G     0   20G   0% /mnt/my-vol
```

## Kubernetes使用GlusterFS

K8s官方文档<https://github.com/kubernetes/examples/tree/master/staging/volumes/glusterfs>

### 添加entrypoint

在gluster服务器上查看服务端口

```
root@10-103-1-11:~# netstat -nptl
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      978/sshd
tcp        0      0 0.0.0.0:49157           0.0.0.0:*               LISTEN      1324/glusterfsd
tcp        0      0 0.0.0.0:24007           0.0.0.0:*               LISTEN      1144/glusterd
tcp6       0      0 :::22                   :::*                    LISTEN      978/sshd
```

我们进程`glusterd`对应的端口`24007`就是服务端口

开始创建entryppint

{% code title="gluster-endpoints.yaml" %}

```yaml
---
kind: Endpoints
apiVersion: v1
metadata:
  labels:
    app: external-glusterfs
  name: glusterfs
  namespace: common
subsets:
- addresses:
  - ip: 10.103.1.11
  - ip: 10.103.1.119
  ports:
  - port: 24007
```

{% endcode %}

### 配置 service

{% code title="gluster-endpoints.yaml" %}

```yaml
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: external-glusterfs
  name: glusterfs
  namespace: common
spec:
  ports:
  - port: 24007
    protocol: TCP
    targetPort: 24007

```

{% endcode %}

### 创建测试 pod

`volumes[0].glusterfs.endpoints`对应的值是上面创建`service`的名字

`volumes[0].glusterfs.path`是上面创建的`volume`名字

```yaml
---
apiVersion: v1
kind: Pod
metadata:
  name: glusterfs-debug
  namespace: common
spec:
  containers:
  - name: glusterfs
    image: nginx
    volumeMounts:
    - mountPath: "/mnt/glusterfs"
      name: glusterfsvol
  volumes:
  - name: glusterfsvol
    glusterfs:
      endpoints: glusterfs
      path: gv0
      readOnly: true
```

查看pod状态，确认启动成功

```
kubectl describe pods/glusterfs
```

### 安装配置Heketi

上面的方式，适合只有很少PV需求的场景，如果有很多Pod需要很多PV，指向Glusterfs不同的卷（或者说不同的目录），那么每次都要手工创建Glusterfs Volume再创建对应的PV就会很麻烦。因此，我们需要一种自动的方式去生成所需资源。K8s支持自定义PVC的`StorageClass`，通过它和`Heketi`配合，就可以自动创建所需的存储资源，不同`Deployment`可以拥有独立目录。

参考文档：

{% embed url="<https://github.com/gluster/gluster-kubernetes/tree/master/docs/examples/dynamic_provisioning_external_gluster>" %}

{% embed url="<https://github.com/heketi/heketi>" %}

添加heketi配置文件：

{% code title="/etc/heketi/heketi.json" %}

```javascript
{
  "_port_comment": "Heketi Server Port Number",
  "port": "8080",

  "_use_auth": "Enable JWT authorization. Please enable for deployment",
  "use_auth": false,

  "_jwt": "Private keys for access",
  "jwt": {
    "_admin": "Admin has access to all APIs",
    "admin": {
      "key": "My Secret"
    },
    "_user": "User only has access to /volumes endpoint",
    "user": {
      "key": "My Secret"
    }
  },

  "_glusterfs_comment": "GlusterFS Configuration",
  "glusterfs": {

    "_executor_comment": "Execute plugin. Possible choices: mock, ssh",
    "executor": "ssh",
    "sshexec": {
      "keyfile": "/etc/heketi/heketi_key",
      "user": "ubuntu",
      "port": "22",
      "fstab": "/etc/fstab",
      "sudo": true
    },

    "_db_comment": "Database file name",
    "db": "/var/lib/heketi/heketi.db"
  }
}
```

{% endcode %}

这里需要注意的`glusterfs.sshexec.sudo`为`true`，否则heketi服务会提示没有权限

**启动`heketi`服务：**&#x4E3A;了方便调试，我们在本地用docker启动服务，生产环境，我们可能会在一台独立电脑上启动服务，或者用K8s在Pod中运行这个服务。

```
docker run --name heketi -v /Users/xiaohui/coding/heketi:/etc/heketi heketi/heketi
```

这里，我们映射了本地目录`/Users/xiaohui/coding/heketi`到容器的`/etc/heketi`

把上面的`heketi.json`文件放到本地目录`/Users/xiaohui/coding/heketi，同时添加`topylogy（Glusterfs Cluster的拓扑图）文件：

{% code title="/etc/heketi/topology.json" %}

```javascript
{
  "clusters": [
    {
      "nodes": [
        {
          "node": {
            "hostnames": {
              "manage": [
                "10.103.1.11"
              ],
              "storage": [
                "10.103.1.11"
              ]
            },
            "zone": 1
          },
          "devices": [
            "/dev/sdb"
          ]
        },
        {
          "node": {
            "hostnames": {
              "manage": [
                "10.103.1.119"
              ],
              "storage": [
                "10.103.1.119"
              ]
            },
            "zone": 1
          },
          "devices": [
            "/dev/sdb"
          ]
        }
      ]
    }
  ]
}
```

{% endcode %}

这个文件的目的是定义Glusterfs服务器地址和磁盘。

然后新开 一个终端，进入heketi容器：

```
docker exec -it heketi bash
```

当我们手工在Ubuntu上安装Glusterfs之后，用heketi-cli加载集群配置时，可能会出现如下错误：

```
[root@b5ff52589fc7 ~]# heketi-cli topology load --json /etc/heketi/topology.json
Creating cluster ... ID: ea58edbcd5880a426771e73d0bd5c0df
	Allowing file volumes on cluster.
	Allowing block volumes on cluster.
	Creating node 10.103.1.11 ... Unable to create node: New Node doesn't have glusterd running
	Creating node 10.103.1.119 ... Unable to create node: New Node doesn't have glusterd running
```

查看Heketi HTTP服务器端日志，会发现如下错误：

```
[negroni] Started GET /queue/a569809c6f54685276f2331aa4891e5f
[negroni] Completed 500 Internal Server Error in 103.1µs
[negroni] Started POST /nodes
[cmdexec] INFO 2018/08/04 10:29:40 Check Glusterd service status in node 10.103.1.11
[cmdexec] DEBUG 2018/08/04 10:29:41 /src/github.com/heketi/heketi/pkg/utils/ssh/ssh.go:174: Host: 10.103.1.11:22 Command: /bin/bash -c 'systemctl status glusterd'
```

这里可以找到原因：`systemctl status glusterd`没有找到服务，是因为Ubuntu安装的gluster服务名是`glusterfs-server`

```
root@10-103-1-11:~# systemctl status glusterfs-server.service
● glusterfs-server.service - LSB: GlusterFS server
   Loaded: loaded (/etc/init.d/glusterfs-server; bad; vendor preset: enabled)
  Drop-In: /etc/systemd/system/glusterfs-server.service.d
           └─override.conf
   Active: active (running) since Sat 2018-08-04 18:26:14 CST; 6min ago
     Docs: man:systemd-sysv-generator(8)
  Process: 9703 ExecStart=/etc/init.d/glusterfs-server start (code=exited, status=0/SUCCESS)
    Tasks: 40
   Memory: 126.6M
      CPU: 1.775s
```

目前，Heketi写死了服务名称，无法配置 <https://github.com/heketi/heketi/blob/6d80b73a799084cd28776e25857225c645756aaf/executors/cmdexec/peer.go#L72>

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LJ3TsV9-qj4Bb_vm0_Q%2F-LJ3VBXLDGkvuTo0eJu0%2Fimage.png?alt=media\&token=7e6aa4b8-9a08-4e7b-9b4e-af2dad4bbe07)

因此，我们只有创建一个systemd别名：`glusterd.service` => `glusterfs-server.service`

方法如下：

```bash
root@10-103-1-11:~# systemctl edit glusterfs-server.service
```

添加内容：

```
[Install]
Alias=glusterd.service
```

获取服务的文件地址`/run/systemd/generator.late/glusterfs-server.service`

```
root@10-103-1-11:~# systemctl cat glusterfs-server.service
# /run/systemd/generator.late/glusterfs-server.service
# Automatically generated by systemd-sysv-generator

[Unit]
Documentation=man:systemd-sysv-generator(8)
SourcePath=/etc/init.d/glusterfs-server
Description=LSB: GlusterFS server
Before=multi-user.target
Before=multi-user.target
Before=multi-user.target
```

创建软连接：

```
ln -sf /run/systemd/generator.late/glusterfs-server.service /etc/systemd/system/glusterd.service
```

启动服务：

```
systemctl daemon-reload
systemctl enable glusterd
systemctl start glusterd
```

现在我们再次加载Glusterfs Cluster Topology 文件

```
heketi-cli topology load --json /etc/heketi/topology.json
[root@c63599638219 /]# heketi-cli topology load --json /etc/heketi/topology.json
	Found node 10.103.1.11 on cluster 2748c838c9e5433c140dd580ce8d92ba
		Adding device /dev/vdb ... OK
	Found node 10.103.1.119 on cluster 2748c838c9e5433c140dd580ce8d92ba
		Adding device /dev/vdb ... OK
```

{% hint style="info" %}
这里需要注意，磁盘/dev/sdb必须是一个空磁盘并且没有被挂载到系统，否则会添加失败。Heketi这么做的目的也是为了保护我们的磁盘数据，万一设置了错误的磁盘，不至于丢失数据。
{% endhint %}

如果磁盘的确已经被挂载或者已经被格式化，我们需要做如下操作：

```
卸载磁盘
umount /dev/sdb
# 删除掉挂载信息
vi /etc/fstab
# 清除磁盘分区
wipefs /dev/vdb -a
```

#### 测试heketi-cli命令

获取集群

```
[root@c63599638219 /]# heketi-cli cluster list
Clusters:
Id:2748c838c9e5433c140dd580ce8d92ba [file][block]
```

获取节点列表

```
[root@c63599638219 /]# heketi-cli node list
Id:783947cc37876e44c1fdee6f1690c1d7	Cluster:2748c838c9e5433c140dd580ce8d92ba
Id:e935a6a52e562477da79d6667401a505	Cluster:2748c838c9e5433c140dd580ce8d92ba
```

创建Volume

```
[root@c63599638219 /]# heketi-cli volume create --size=10 --replica=2  --name my-vol
Name: my-vol
Size: 10
Volume Id: 66f29b294df96b378548f10ee898eaf5
Cluster Id: 2748c838c9e5433c140dd580ce8d92ba
Mount: 10.103.1.119:my-vol
Mount Options: backup-volfile-servers=10.103.1.11
Block: false
Free Size: 0
Block Volumes: []
Durability Type: replicate
Distributed+Replica: 2
```

{% hint style="info" %}
创建Volume时可能会出现错误“usr sbin thin\_check no such file or directory”，需要在每台Glusterfs服务器上安装：apt install thin-provisioning-tools
{% endhint %}

获取卷列表

```
[root@c63599638219 /]# heketi-cli volume list
Id:66f29b294df96b378548f10ee898eaf5    Cluster:2748c838c9e5433c140dd580ce8d92ba    Name:my-vol
```

获取拓扑信息

```
[root@c63599638219 /]# heketi-cli topology info

Cluster Id: 2748c838c9e5433c140dd580ce8d92ba

    File:  true
    Block: true

    Volumes:

	Name: my-vol
	Size: 10
	Id: 66f29b294df96b378548f10ee898eaf5
	Cluster Id: 2748c838c9e5433c140dd580ce8d92ba
	Mount: 10.103.1.119:my-vol
	Mount Options: backup-volfile-servers=10.103.1.11
	Durability Type: replicate
	Replica: 2
	Snapshot: Disabled

		Bricks:
			Id: 44b8c2bdbd04fbeadd197e10a25701b0
			Path: /var/lib/heketi/mounts/vg_26cea42d1a71151288db4cab8477bc86/brick_44b8c2bdbd04fbeadd197e10a25701b0/brick
			Size (GiB): 10
			Node: e935a6a52e562477da79d6667401a505
			Device: 26cea42d1a71151288db4cab8477bc86

			Id: de363568400f13f28e0bb83845c549bd
			Path: /var/lib/heketi/mounts/vg_75a66dbe5b2c64db3678f052837fc9c2/brick_de363568400f13f28e0bb83845c549bd/brick
			Size (GiB): 10
			Node: 783947cc37876e44c1fdee6f1690c1d7
			Device: 75a66dbe5b2c64db3678f052837fc9c2


    Nodes:

	Node Id: 783947cc37876e44c1fdee6f1690c1d7
	State: online
	Cluster Id: 2748c838c9e5433c140dd580ce8d92ba
	Zone: 1
	Management Hostnames: 10.103.1.119
	Storage Hostnames: 10.103.1.119
	Devices:
		Id:75a66dbe5b2c64db3678f052837fc9c2   Name:/dev/vdb            State:online    Size (GiB):99      Used (GiB):10      Free (GiB):89
			Bricks:
				Id:de363568400f13f28e0bb83845c549bd   Size (GiB):10      Path: /var/lib/heketi/mounts/vg_75a66dbe5b2c64db3678f052837fc9c2/brick_de363568400f13f28e0bb83845c549bd/brick

	Node Id: e935a6a52e562477da79d6667401a505
	State: online
	Cluster Id: 2748c838c9e5433c140dd580ce8d92ba
	Zone: 1
	Management Hostnames: 10.103.1.11
	Storage Hostnames: 10.103.1.11
	Devices:
		Id:26cea42d1a71151288db4cab8477bc86   Name:/dev/vdb            State:online    Size (GiB):99      Used (GiB):10      Free (GiB):89
			Bricks:
				Id:44b8c2bdbd04fbeadd197e10a25701b0   Size (GiB):10      Path: /var/lib/heketi/mounts/vg_26cea42d1a71151288db4cab8477bc86/brick_44b8c2bdbd04fbeadd197e10a25701b0/brick
[root@c63599638219 /]#
```

获取磁盘信息

```
[root@c63599638219 /]# heketi-cli device info 26cea42d1a71151288db4cab8477bc86
Device Id: 26cea42d1a71151288db4cab8477bc86
Name: /dev/vdb
State: online
Size (GiB): 99
Used (GiB): 10
Free (GiB): 89
Bricks:
Id:44b8c2bdbd04fbeadd197e10a25701b0   Size (GiB):10      Path: /var/lib/heketi/mounts/vg_26cea42d1a71151288db4cab8477bc86/brick_44b8c2bdbd04fbeadd197e10a25701b0/brick
```

删除卷

```
[root@c63599638219 /]# heketi-cli volume delete 66f29b294df96b378548f10ee898eaf5
Volume 66f29b294df96b378548f10ee898eaf5 deleted
```


# 日志管理

## Fluentd

> 特点：轻量，作为ruby Gem，可以收集主流日志，输出到主流存储系统。插件丰富，配置简单

>

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LAIHAn_zUv_aUT-mJeB%2F-LAIHJfigULkPVtnyZ4y%2Fimage.png?alt=media\&token=719b3906-520d-4937-b4a4-fb84bdf17982)


# Fluentd

### Installing Fluentd Using Ruby Gem

>

### Docker 镜像

>

### 下载

>


# VirtualBox

## Ubuntu下安装VirtualBox

&#x20; 参考文档：

>

```bash
wget -q https://www.virtualbox.org/download/oracle_vbox_2016.asc -O- | sudo apt-key add -
echo deb https://mirrors.tuna.tsinghua.edu.cn/virtualbox/apt/ xenial contrib|sudo tee /etc/apt/sources.list.d/virtualbox.list
sudo apt-get install virtualbox virtualbox-ext-pack -y 
```

{% hint style="info" %}
注意：这里需要安装`virtualbox-ext-pack`，否则无法虚拟机开启远程桌面，就无法安装系统
{% endhint %}

## 命令行安装Ubuntu系统

参考文档：

>

### 下载Ubuntu安装盘ISO镜像

```bash
mkdir -p ~/VirtualBox\ VMs/
curl --progress-bar -o ~/VirtualBox\ VMs/ubuntu-16.04.4-server-amd64.iso https://mirrors.tuna.tsinghua.edu.cn/ubuntu-releases/16.04/ubuntu-16.04.4-server-amd64.iso
```

### 创建虚拟机

```bash
# 定义个变量，要创建的虚拟机名称
vm=node1
vboxmanage createvm -name ${vm} -register
```

{% code title="输出内容：" %}

```bash
Virtual machine 'node1' is created and registered.
UUID: 3d26d492-880a-43c2-ac47-6764abc5ef1d
Settings file: '/home/xiaohui/VirtualBox VMs/node1/node1.vbox'
```

{% endcode %}

### &#x20;配置虚拟机硬件信息

```bash
# 查看支持的操作系统，这里我们将记录下Ubuntu_64是我们需要安装的操作系统类型
vboxmanage list ostypes
```

{% code title="输出内容：" %}

```bash
...
ID:          Ubuntu
Description: Ubuntu (32-bit)
Family ID:   Linux
Family Desc: Linux
64 bit:      false

ID:          Ubuntu_64
Description: Ubuntu (64-bit)
Family ID:   Linux
Family Desc: Linux
64 bit:      true
...
```

{% endcode %}

```bash
# 查看主机上的IP地址，得到上网网卡的接口名称，这里是 enp8s0
ip addr
```

{% code title="输出内容：" %}

```bash
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: enp8s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 08:57:00:f1:7d:c6 brd ff:ff:ff:ff:ff:ff
    inet 10.0.1.16/24 brd 10.0.1.255 scope global enp8s0
       valid_lft forever preferred_lft forever
    inet6 2002:b695:9fa8:0:a57:ff:fef1:7dc6/64 scope global mngtmpaddr dynamic
       valid_lft 14300sec preferred_lft 3500sec
    inet6 fe80::a57:ff:fef1:7dc6/64 scope link
       valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
    link/ether 02:42:fe:ba:2f:f6 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
    inet6 fe80::42:feff:feba:2ff6/64 scope link
       valid_lft forever preferred_lft forever
494: vboxnet0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 0a:00:27:00:00:00 brd ff:ff:ff:ff:ff:ff
```

{% endcode %}

```bash
# 修改虚拟机的硬件配置
vboxmanage modifyvm ${vm} \
  --memory 3072 \
  --ostype Ubuntu_64 \
  --acpi on \
  --cpus 2 \
  --boot1 dvd \
  --nic1 bridged \
  --bridgeadapter1 enp8s0
```

{% code title="参数说明" %}

```bash
--memory 3072 # 内存为3G
--ostype Ubuntu_64 # 操作系统类型为Ubuntu 64
--acpi on # 开启ACPI电源控制
--cpus 2 # CPU核心数为双核
--boot1 dvd # 第一启动设备为DVD光驱
--nic1 bridged # 网路类型为桥接模式
--bridgeadapter1 enp8s0 # 网络接口为 enp8s0
```

{% endcode %}

```bash
# 创建一个硬盘, 磁盘大小为64G
vboxmanage createhd --filename ~/VirtualBox\ VMs/${vm}/os.vdi --size 64000
# 添加一个SATA控制器
vboxmanage storagectl ${vm} --name "SATA Controller" --add sata --controller IntelAHCI
# 把创建的硬盘挂载到虚拟机
vboxmanage storageattach ${vm} --storagectl "SATA Controller" --port 0 --device 0 --type hdd --medium ~/VirtualBox\ VMs/${vm}/os.vdi
# 创建一个IDE控制器
vboxmanage storagectl ${vm} --name "IDE Controller" --add ide
# 把下载的Ubuntu镜像挂载到虚拟机
vboxmanage storageattach ${vm} --storagectl "IDE Controller" --port 0 --device 0 --type dvddrive --medium ~/VirtualBox\ VMs/ubuntu-16.04.4-server-amd64.iso
```

### 启动虚拟机

```bash
# 开启远程桌面管理
vboxmanage modifyvm ${vm} --vrdeport 3390 --vrde on
# 启动
vboxmanage startvm ${vm} --type headless

# 查看启动虚拟机列表
xiaohui@tanmer-dev:~$ vboxmanage list runningvms
"node1" {3d26d492-880a-43c2-ac47-6764abc5ef1d}

# 查看虚拟机信息
xiaohui@tanmer-dev:~$ vboxmanage showvminfo $vm
Name:            node1
Groups:          /
Guest OS:        Ubuntu (64-bit)
UUID:            3d26d492-880a-43c2-ac47-6764abc5ef1d
Config file:     /home/xiaohui/VirtualBox VMs/node1/node1.vbox
Snapshot folder: /home/xiaohui/VirtualBox VMs/node1/Snapshots
Log folder:      /home/xiaohui/VirtualBox VMs/node1/Logs
Hardware UUID:   3d26d492-880a-43c2-ac47-6764abc5ef1d
Memory size:     3072MB
Page Fusion:     off
VRAM size:       8MB
CPU exec cap:    100%
HPET:            off
Chipset:         piix3
Firmware:        BIOS
Number of CPUs:  2
PAE:             on
Long Mode:       on
Triple Fault Reset: off
APIC:            on
X2APIC:          off
CPUID Portability Level: 0
CPUID overrides: None
Boot menu mode:  message and menu
Boot Device (1): DVD
Boot Device (2): DVD
Boot Device (3): HardDisk
Boot Device (4): Not Assigned
ACPI:            on
IOAPIC:          off
BIOS APIC mode:  APIC
Time offset:     0ms
RTC:             local time
Hardw. virt.ext: on
Nested Paging:   on
Large Pages:     off
VT-x VPID:       on
VT-x unr. exec.: on
Paravirt. Provider: Default
Effective Paravirt. Provider: KVM
State:           running (since 2018-04-18T16:11:32.153000000)
Monitor count:   1
3D Acceleration: off
2D Video Acceleration: off
Teleporter Enabled: off
Teleporter Port: 0
Teleporter Address:
Teleporter Password:
Tracing Enabled: off
Allow Tracing to Access VM: off
Tracing Configuration:
Autostart Enabled: off
Autostart Delay: 0
Default Frontend:
Storage Controller Name (0):            SATA Controller
Storage Controller Type (0):            IntelAhci
Storage Controller Instance Number (0): 0
Storage Controller Max Port Count (0):  30
Storage Controller Port Count (0):      30
Storage Controller Bootable (0):        on
Storage Controller Name (1):            IDE Controller
Storage Controller Type (1):            PIIX4
Storage Controller Instance Number (1): 0
Storage Controller Max Port Count (1):  2
Storage Controller Port Count (1):      2
Storage Controller Bootable (1):        on
SATA Controller (0, 0): /home/xiaohui/VirtualBox VMs/node1/os.vdi (UUID: 76208d22-3257-4496-898a-2bd17e930b43)
IDE Controller (0, 0): /home/xiaohui/VirtualBox VMs/ubuntu-16.04.4-server-amd64.iso (UUID: d724e290-e032-42d6-91eb-03b8e4a18004)
NIC 1:           MAC: 0800276585BE, Attachment: Bridged Interface 'enp8s0', Cable connected: on, Trace: off (file: none), Type: Am79C973, Reported speed: 0 Mbps, Boot priority: 0, Promisc Policy: deny, Bandwidth group: none
NIC 2:           disabled
NIC 3:           disabled
NIC 4:           disabled
NIC 5:           disabled
NIC 6:           disabled
NIC 7:           disabled
NIC 8:           disabled
Pointing Device: PS/2 Mouse
Keyboard Device: PS/2 Keyboard
UART 1:          disabled
UART 2:          disabled
UART 3:          disabled
UART 4:          disabled
LPT 1:           disabled
LPT 2:           disabled
Audio:           enabled (Driver: ALSA, Controller: AC97, Codec: STAC9700)
Audio playback:  disabled
Audio capture: disabled
Clipboard Mode:  disabled
Drag and drop Mode: disabled
Session name:    headless
Video mode:      640x480x16 at 0,0 enabled
VRDE:            enabled (Address 0.0.0.0, Ports 3390, MultiConn: off, ReuseSingleConn: off, Authentication type: null)
Video redirection: disabled
USB:             disabled
EHCI:            disabled
XHCI:            disabled

USB Device Filters:

<none>

Available remote USB devices:

<none>

Currently Attached USB Devices:

<none>

Bandwidth groups:  <none>

Shared folders:  <none>

VRDE Connection:    not active
Clients so far:     0

Capturing:          not active
Capture audio:      not active
Capture screens:    0
Capture file:       /home/xiaohui/VirtualBox VMs/node1/node1.webm
Capture dimensions: 1024x768
Capture rate:       512 kbps
Capture FPS:        25
Capture options:

Guest:

Configured memory balloon size:      0 MB
OS type:                             Ubuntu_64
Additions run level:                 0

Guest Facilities:

No active facilities.
```

### 连接虚拟机远程桌面

映射主机的3390端口到本地电脑，然后用 RDP 远程桌面连接工具连接虚拟机远程桌面，Mac 系统推荐使用 [Remotix](http://www.ifunmac.com/2017/10/remotix-5/)

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LAOefBJsoI1CmIp5a_H%2F-LAOfHrQLIhTj_jKr6-R%2Fimage.png?alt=media\&token=0681b8b9-f468-438e-8b84-3c3cb6b0ac80)

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LAOefBJsoI1CmIp5a_H%2F-LAOfMfbG9NeJKYe99sr%2Fimage.png?alt=media\&token=95a30137-df9e-4a8a-b66a-475176e495fd)

如何装Ubuntu，这里略过...

### 卸载Ubuntu安装ISO镜像

```bash
vboxmanage storageattach $vm --storagectl "IDE Controller" --port 0 --device 0 --type dvddrive --medium none
```

### 关闭远程桌面

```bash
# 运行状态，关闭远程桌面
vboxmanage controlvm $vm vrde off
# 运行状态，开启远程桌面
vboxmanage controlvm $vm vrde on
# 修改配置，关闭远程桌面（需要关闭虚拟机）
vboxmanage modifyvm ${vm} --vrde off
# 修改配置，开启远程桌面（需要关闭虚拟机）
vboxmanage modifyvm ${vm} --vrde on

```

### 备份虚拟机系统(建立快照)

```bash
vboxmanage snapshot $vm take backup-20180418-001
```

### 还原虚拟机系统(恢复快照)

```bash
vboxmanage snapshot $vm restore backup-20180418-001
```

### 查看快照列表

```bash
xiaohui@tanmer-dev:~$ vboxmanage snapshot node1 list
   Name: init-ubuntu-20180418 (UUID: 1960cff6-e827-4a8f-a0df-8bba2b580f23) *
```

### 关闭虚拟机

```bash
vboxmanage controlvm $vm poweroff
```

### 注销虚拟机

```bash
# 注销，不删除数据
vboxmanage unregister $vm
# 注销，并删除数据
vboxmanage unregister $vm --delete
```

### 克隆创建的Ubuntu系统

当我们需要创建多个虚拟机Ubuntu系统时，我们没有必要每次都重复上面的安装步骤。我们可以用VirtualBox的克隆功能，快速复制个Ubuntu系统

通过快照克隆系统

```bash
# 通过node1的init-ubuntu-20180814快照克隆系统
vboxmanage clonevm node1 --snapshot init-ubuntu-20180418 --name node2 --register
# 启动node2
vboxmanage startvm node2 --type headless
```

从`node1`克隆为`node2`之后，她的hostname没有改变，这里需要修改：

```bash
sudo sed -i 's/node1/node2/' /etc/hosts
sudo sed -i 's/node1/node2/' /etc/hostname
sudo hostname node2
```

IP地址的设置也需要修改

```bash
sudo vi /etc/networking/interfaces
```

启动克隆后的虚拟机，如果是Bridged网路可能会无法上网。可以通过一下方式解决：

```bash
vboxmanage controlvm node2 poweroff
vboxmanage modifyvm node2 --nic1 none
vboxmanage modifyvm node2 --nic1 bridged
vboxmanage startvm node2 --type headless
```


# 工具软件


# Alfred

## 自动通过翻墙浏览器上网

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LAahF0e8MtDDQZsc8k-%2F-LAai1B_-MgYhPm3zzJ2%2Fimage.png?alt=media\&token=941a253a-6323-4928-a1d4-4bd41136b3f3)

### 原理

通过Chrome浏览器的`Profile`功能，专门新建一个`翻墙`Profile，配置好翻墙代理服务器。Alfred通过命令行打开Chrome浏览器，传入参数，让Chrome选择对应的Profile，打开网站。

{% code title="打开Chrome浏览器，指定 Profile的命令行" %}

```bash
"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" "http://www.google.com" --profile-directory="Profile 4"
```

{% endcode %}

这里唯一的问题就是`--profile-directory`填什么。这里我们可以借助 [Alfred Chrome](https://github.com/ShogunPanda/alfred-chrome) 得到浏览器Profile对应的目录是什么

### 安装Alfred Chrome Workflow

打开 <http://www.packal.org/workflow/alfred-chrome>

下载 <https://github.com/packal/repository/raw/master/it.cowtech.alfred.chrome/alfred_chrome.alfredworkflow>

双击下载的文件，workflow会自动添加到Alfred

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LAahF0e8MtDDQZsc8k-%2F-LAaovDXFtLKgMDBkBpP%2Fimage.png?alt=media\&token=9f857503-e665-46a7-a98b-5a204d16877d)

双击cr

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LAahF0e8MtDDQZsc8k-%2F-LAap92MyP7hd9WZmflL%2Fimage.png?alt=media\&token=ed308b15-2d6f-4898-bbcf-8bcd12ea7bcf)

点下面圆圈处，会打开Finder

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LAahF0e8MtDDQZsc8k-%2F-LAapTdy7C6ackzCP5Wx%2Fimage.png?alt=media\&token=8d43d4a9-2607-4b85-bb0e-b9d70fc8a923)

### 找出翻墙Profile所在的目录名

在bash中执行alfred-chrome文件，会输出JSON文件，从中找出自己配置的翻墙profile的目录名

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LAahF0e8MtDDQZsc8k-%2F-LAaqBAnygXmk52R0dFM%2Fimage.png?alt=media\&token=e722f63b-1e09-4ac1-84c6-fb1e27e63309)

这里，我的翻墙profile目录是`Profile 4`

下载我制作好的workflow文件 [gfw.alfredworkflow](https://github.com/tanmer/tanmer.github.io/raw/master/.gitbook/assets/gfw.alfredworkflow) , 打开workflows, 双击`/bin/bash`节点，修改`Profile 4`为你自己的翻墙目录

```
"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" $1 --profile-directory="Profile 4"
```

### 开始使用

Alfred搜索条中试试输入`fq`,`gem`,`github`,`superu`,`stackof`


# 代码版本控制工具

## Git

### Github Desktop

{% embed url="<https://desktop.github.com/>" %}

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LAMhrypakM1OaJhVK1P%2F-LAMibzS5EEBfBc6aKGs%2Fimage.png?alt=media\&token=d5fd9dcf-acaa-4c23-a5ee-4d24da3b1dad)

## SVN

### Cornerstone

{% embed url="<http://www.ifunmac.com/2016/09/cornerstone-3-0-3/>" %}

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LAMhrypakM1OaJhVK1P%2F-LAMip0SzGfQ8d1RFP3g%2Fimage.png?alt=media\&token=16a452ad-68aa-4e24-b992-1893757c7670)


# Atom

## 配置技巧

### 显示.gitignore文件

不知道从哪个版本开始，Atom默认不显示.gitignore中忽略掉的文件。通过以下方法可以显示：

设置中->`Packages`，搜索`tree-view`，点`Settings`，取消勾选`Hide VCS Ignored Files`

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LAQscmqyzvRHDufU1y7%2F-LAQtnr0zfEVQfsEFoB6%2Fimage.png?alt=media\&token=2f048e67-ab38-4e25-9f26-1760a262f684)


# Bash Shell

## 通用

### 如何改变指定用户的登录shell

```bash
chsh <用户名> -s <新shell>
chsh xiaohui -s /bin/sh
```

### 标准输出和错误输出同时重定向到同一位置

方法一：

```bash
2>&1 (如# ls /usr/share/doc > out.txt 2>&1 )
```

方法二：

```bash
&> (如# ls /usr/share/doc &> out.txt )
```

## 条件判断

```bash
if [ 条件 ]
then
命令1
命令2
…..
else
if [ 条件 ]
then
命令1
命令2
….
else
命令1
命令2
…..
fi
fi
```

### 比较两个数字

```bash
#!/bin/bash
x=10
y=20
if [ $x -gt $y ]
then
echo "x is greater than y"
else
echo "y is greater than x"
fi
```

### 如何测试文件

```
Test         用法
-d 文件名    如果文件存在并且是目录，返回true
-e 文件名    如果文件存在，返回true
-f 文件名    如果文件存在并且是普通文件，返回true
-r 文件名    如果文件存在并可读，返回true
-s 文件名    如果文件存在并且不为空，返回true
-w 文件名    如果文件存在并可写，返回true
-x 文件名    如果文件存在并可执行，返回true
```

比如测试文件是否存在：

```bash
printf "正在复制 config/application.example.yml 到 config/application.yml ..."
if [ -e config/application.yml ]; then
  echo "已经存在，跳过复制"
else
  cp config/application{.example,}.yml
fi
```

###


# Vim

## Cheatsheet&#x20;

<https://vim.rtorr.com/>

## 快捷键

### 代码块复制

⇧+v 进入可视行模式

按上下键选择行，然后按`y`, 走到要粘贴的行，按`p`

### 多行注释

在需要注释的第一行，按`Ctrl` +`V，`然后按向下方向键，走到想要注释的最后一行，按大写`i`进入插入模式，输入`#` ，按`Esc`退出插入模式，再按一次`Esc`即可实现多行注释。

## 配置详解

{% code title="\~/.vimrc" %}

```
" 不要使用vi的键盘模式，而是vim自己的  
set nocompatible  
  
" 语法高亮  
set syntax=on  
  
" 去掉输入错误的提示声音  
set noeb  
  
" 在处理未保存或只读文件的时候，弹出确认  
set confirm  
  
" 自动缩进  
set autoindent  
set cindent  
  
" Tab键的宽度  
set tabstop=4  
  
" 统一缩进为4  
set softtabstop=4  
set shiftwidth=4  
  
" 不要用空格代替制表符  
set noexpandtab  
  
" 在行和段开始处使用制表符  
set smarttab  
  
" 显示行号  
set number  
  
" 历史记录数  
set history=1000  
  
"禁止生成临时文件  
set nobackup  
set noswapfile  
  
"搜索忽略大小写  
set ignorecase  
  
"搜索逐字符高亮  
set hlsearch  
set incsearch  
  
"行内替换  
set gdefault  
  
"编码设置  
set enc=utf-8  
set fencs=utf-8,ucs-bom,shift-jis,gb18030,gbk,gb2312,cp936  
  
"语言设置  
set langmenu=zh_CN.UTF-8  
set helplang=cn  
  
" 我的状态行显示的内容（包括文件类型和解码）  
set statusline=%F%m%r%h%w\ [FORMAT=%{&ff}]\ [TYPE=%Y]\ [POS=%l,%v][%p%%]\ %{strftime(\"%d/%m/%y\ -\ %H:%M\")}  
"set statusline=[%F]%y%r%m%*%=[Line:%l/%L,Column:%c][%p%%]  
  
" 总是显示状态行  
set laststatus=2  
  
" 在编辑过程中，在右下角显示光标位置的状态行  
set ruler             
  
" 命令行（在状态行下）的高度，默认为1，这里是2  
set cmdheight=2  
  
" 侦测文件类型  
filetype on  
  
" 载入文件类型插件  
filetype plugin on  
  
" 为特定文件类型载入相关缩进文件  
filetype indent on  
  
" 保存全局变量  
set viminfo+=!  
  
" 带有如下符号的单词不要被换行分割  
set iskeyword+=_,$,@,%,#,-  
  
" 字符间插入的像素行数目  
set linespace=0  
  
" 增强模式中的命令行自动完成操作  
set wildmenu  
  
" 使回格键（backspace）正常处理indent, eol, start等  
set backspace=2  
  
" 允许backspace和光标键跨越行边界  
set whichwrap+=<,>,h,l  
  
" 可以在buffer的任何地方使用鼠标（类似office中在工作区双击鼠标定位）  
set mouse=a  
set selection=exclusive  
set selectmode=mouse,key  
  
" 通过使用: commands命令，告诉我们文件的哪一行被改变过  
set report=0  
  
" 启动的时候不显示那个援助索马里儿童的提示  
set shortmess=atI  
  
" 在被分割的窗口间显示空白，便于阅读  
set fillchars=vert:\ ,stl:\ ,stlnc:\  
  
" 高亮显示匹配的括号  
set showmatch  
  
" 匹配括号高亮的时间（单位是十分之一秒）  
set matchtime=5  
  
" 光标移动到buffer的顶部和底部时保持3行距离  
set scrolloff=3  
  
" 为C程序提供自动缩进  
set smartindent  
  
" 只在下列文件类型被侦测到的时候显示行号，普通文本文件不显示  
if has("autocmd")  
   autocmd FileType xml,html,c,cs,java,perl,shell,bash,cpp,python,vim,php,ruby set number  
   autocmd FileType xml,html vmap <C-o> <ESC>'<i<!--<ESC>o<ESC>'>o-->  
   autocmd FileType java,c,cpp,cs vmap <C-o> <ESC>'<o/*<ESC>'>o*/  
   autocmd FileType html,text,php,vim,c,java,xml,bash,shell,perl,python setlocal textwidth=100  
   autocmd Filetype html,xml,xsl source $VIMRUNTIME/plugin/closetag.vim  
   autocmd BufReadPost *  
      \ if line("'\"") > 0 && line("'\"") <= line("$") |  
      \   exe "normal g`\"" |  
      \ endif  
endif " has("autocmd")  
  
" F5编译和运行C程序，F6编译和运行C++程序  
" 请注意，下述代码在windows下使用会报错  
" 需要去掉./这两个字符  
  
" C的编译和运行  
map <F5> :call CompileRunGcc()<CR>  
func! CompileRunGcc()  
exec "w"  
exec "!gcc % -o %<"  
exec "! ./%<"  
endfunc  
  
" C++的编译和运行  
map <F6> :call CompileRunGpp()<CR>  
func! CompileRunGpp()  
exec "w"  
exec "!g++ % -o %<"  
exec "! ./%<"  
endfunc  
  
" 能够漂亮地显示.NFO文件  
set encoding=utf-8  
function! SetFileEncodings(encodings)  
    let b:myfileencodingsbak=&fileencodings  
    let &fileencodings=a:encodings  
endfunction  
function! RestoreFileEncodings()  
    let &fileencodings=b:myfileencodingsbak  
    unlet b:myfileencodingsbak  
endfunction  
  
au BufReadPre *.nfo call SetFileEncodings('cp437')|set ambiwidth=single  
au BufReadPost *.nfo call RestoreFileEncodings()  
  
" 高亮显示普通txt文件（需要txt.vim脚本）  
au BufRead,BufNewFile *  setfiletype txt  
  
" 用空格键来开关折叠  
set foldenable  
set foldmethod=manual  
nnoremap <space> @=((foldclosed(line('.')) < 0) ? 'zc' : 'zo')<CR>  
  
" minibufexpl插件的一般设置  
let g:miniBufExplMapWindowNavVim = 1  
let g:miniBufExplMapWindowNavArrows = 1  
let g:miniBufExplMapCTabSwitchBufs = 1  
let g:miniBufExplModSelTarget = 1   
```

{% endcode %}

## 安装Vundle插件管理工具

用Vim，第一件事就是下载插件管理器 [Vundle](https://github.com/VundleVim/Vundle.vim) (vim bundler)，有了他，就能让Vim用上丰富的插件

```bash
git clone https://github.com/VundleVim/Vundle.vim.git ~/.vim/bundle/Vundle.vim
```

然后是修改`.vimrc`配置文件文件，使用`Vundle`

```
set nocompatible              " be iMproved, required
filetype off                  " required
syntax  on

" set the runtime path to include Vundle and initialize
set rtp+=~/.vim/bundle/Vundle.vim
call vundle#begin()
" alternatively, pass a path where Vundle should install plugins
"call vundle#begin('~/some/path/here')

" let Vundle manage Vundle, required
Plugin 'VundleVim/Vundle.vim'


" All of your Plugins must be added before the following line
call vundle#end()            " required
filetype plugin indent on    " required
" To ignore plugin indent changes, instead use:
"filetype plugin on
"
" Brief help
" :PluginList       - lists configured plugins
" :PluginInstall    - installs plugins; append `!` to update or just :PluginUpdate
" :PluginSearch foo - searches for foo; append `!` to refresh local cache
" :PluginClean      - confirms removal of unused plugins; append `!` to auto-approve removal
"
" see :h vundle for more details or wiki for FAQ
" Put your non-Plugin stuff after this line
```

### 通过Vundle安装插件

{% code title="\~/.vimrc" %}

```
Plugin 'scrooloose/nerdtree'
```

{% endcode %}

{% code title="vim 编辑器中执行" %}

```
:source %
:PlugInstall
```

{% endcode %}

## 常用插件

### NERDTree

> <https://github.com/scrooloose/nerdtree>

NERDTree是Vim的文件系统浏览器，通过这个插件，用户能够可视化地查看目录结构，快速打开，查看和编辑文件。该插件还可以通过特定的API自定义功能映射。

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LAYjIH-g635ZYwPPVA8%2F-LAYn9iMj6IlBqVafm9B%2Fimage.png?alt=media\&token=313aa068-e9a3-4e7b-8045-ec4890732fd2)

#### 如何安装

编辑`.vimrc`，添加`Plugin 'scrooloose/nerdtree'`

{% code title="\~/.vimrc" %}

```
" let Vundle manage Vundle, required
Plugin 'VundleVim/Vundle.vim'

Plugin 'scrooloose/nerdtree'

" All of your Plugins must be added before the following line
```

{% endcode %}

安装插件

```
:source %
:PluginInstall
```

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LAYjIH-g635ZYwPPVA8%2F-LAYpy5tu4smDCdxm8in%2Fimage.png?alt=media\&token=2629ad2a-8c33-4db6-91bc-f31ee06bcc53)


# fzf(Fuzzy Finder)

<https://github.com/junegunn/fzf>

这是一个超快的文件路径模糊搜索工具，功能就像`Sublime Text`的`ctrl+p`,输入`acacrb`就能搜索出文件`app/controller/application_controller.rb`

## 安装

```bash
brew install fzf
```

## Vim支持fzf

{% code title="\~/.vimrc" %}

```
set rtp+=/usr/local/opt/fzf
" 设置快捷键\+f开启搜索
map <Leader>f :FZF<CR>
```

{% endcode %}

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LAZ-O4vhl6pnVuDYGpT%2F-LAZ4MFsrCk3h_ovkNLO%2Fimage.png?alt=media\&token=f68db56e-29f4-46f3-8ecd-6ef59924f5be)


# Gitlab

## 维护

### 解除IP屏蔽

Gitlab默认开启了[rack-attack](https://github.com/kickstarter/rack-attack/blob/master/README.md)，当Gitlab Runner执行CI，频繁PUSH Docker镜像时，会造成频繁登录，然后IP被Gitlab禁止访问。

解决办法是把Gitlab Runner的IP加入到白名单，然后被禁止的IP从Redis中清除。

{% code title="/etc/gitlab/gitlab.rb" %}

```ruby
 gitlab_rails['rack_attack_git_basic_auth'] = {
   'enabled' => true,
   'ip_whitelist' => ["127.0.0.1"],
   'maxretry' => 10,
   'findtime' => 60,
   'bantime' => 3600
 }
 
 # gitlab_rails['rack_attack_protected_paths'] = [
#   '/users/password',
#   '/users/sign_in',
#   '/api/#{API::API.version}/session.json',
#   '/api/#{API::API.version}/session',
#   '/users',
#   '/users/confirmation',
#   '/unsubscribes/',
#   '/import/github/personal_access_token'
# ]
```

{% endcode %}

查看屏蔽日志：

```bash
grep "Rack_Attack" /var/log/gitlab/gitlab-rails/production.log

    Rack_Attack: blacklist 182.149.159.201 POST /api/v4/runners
```

进入Redis CLI：

```bash
/opt/gitlab/embedded/bin/redis-cli -h 10.9.61.112
```

查看被禁止的IP：

```
keys *rack::attack*
```

解除禁止：

```
del cache:gitlab:rack::attack:allow2ban:ban:182.149.159.201
```

### 每周清理Registry数据

当Gitlab集成CI/CD和Docker Registry之后，每次部署时编译Docker镜像都会产生很多文件，时间长了，一些老的镜像过时了，我们会在Gitlab的Registry页面删除他们，可是这个删除动作不会删除服务器上Docker Registry所在目录的数据。`gitlab-ctl`提供了命令行清理Registry的垃圾数据：

{% code title="/etc/crontab" %}

```
0  0    * * 1   root    gitlab-ctl registry-garbage-collect
```

{% endcode %}


# Ubuntu

## 修改apt源

```bash
sudo sed -i 's!http://us.archive.ubuntu.com!https://mirrors.tuna.tsinghua.edu.cn!' /etc/apt/sources.list
```

## 配置

### 允许root远程登录

Ubuntu默认是不允许root远程登录的，我们可以修改`PermitRootLogin`的值，改变root登录方式。

```
vi /etc/ssh/sshd_config
```

```
PermitRootLogin yes # 允许root用密码和Public Key方式登录
PermitRootLogin prohibit-password # 
PermitRootLogin without-password # 只允许root用public key认证方式登录
PermitRootLogin no # 不允许root登录
```

## 翻墙技巧

### 傻瓜式

<https://github.com/googlehosts/hosts>

下载hosts文件，内容添加到`/etc/hosts`

### 进阶式

下载上面仓库的[dnsmasq.conf](https://github.com/googlehosts/hosts/raw/master/hosts-files/dnsmasq.conf)文件，安装dnsmasq服务

```bash
sudo apt-get install dnsmasq
sudo mv ~/dnsmasq.conf /etc/dnsmasq.conf
```

## 软件

### arp

> 查看局域网内IP的Mac地址

```bash
$ arp
Address                  HWtype  HWaddress           Flags Mask            Iface
10.10.0.11               ether   00:04:ff:ff:ff:d0   C                     eth0
10.10.0.16               ether   00:04:ff:ff:ff:a6   C                     eth0
raspbmc.local            ether   00:1f:ff:ff:ff:9c   C                     eth0
10.10.0.19               ether   00:04:ff:ff:ff:c9   C                     eth0
10.10.0.12               ether   bc:f5:ff:ff:ff:93   C                     eth0
10.10.0.17               ether   00:04:ff:ff:ff:57   C                     eth0
10.10.0.1                ether   20:4e:ff:ff:ff:30   C                     eth0
HPF2257E.local           ether   a0:b3:ff:ff:ff:7e   C                     eth0
10.10.0.15               ether   00:04:ff:ff:ff:b9   C                     eth0
tim                      ether   00:22:ff:ff:ff:af   C                     eth0
10.10.0.13               ether   60:be:ff:ff:ff:e0   C                     eth0
```

### iperf

> 带宽测试

```bash
apt install iperf
```

测试方法：准备2台 Linux 云主机（例如CentOS 6.5），一台做server，一台做client，云主机推荐配置为 2核4G

server端命令如下：

```bash
iperf -s
```

client端命令如下：

```
iperf -c 10.x.x.x(server内网IP) -i 2 -t 30 -P 50
```

### ping

> 检查IP知否联网

```bash
apt install iputils-ping -y
```

### ps

> 查看进程

```bash
apt install procps -y
```

### nmap

> 查看局域网内IP和Mac地址信息

```bash
apt install nmap
```

```bash
root@192-168-1-141:~# nmap -sn 192.168.1.0/24

Starting Nmap 7.01 ( https://nmap.org ) at 2018-04-25 16:52 CST
Nmap scan report for 192.168.1.1
Host is up (0.00046s latency).
MAC Address: FA:FF:FF:FF:FF:FF (Unknown)
Nmap scan report for 192.168.1.127
Host is up (-0.099s latency).
MAC Address: 52:54:00:66:42:AF (QEMU virtual NIC)
Nmap scan report for 192.168.1.185
Host is up (-0.100s latency).
MAC Address: 52:54:00:3F:18:04 (QEMU virtual NIC)
Nmap scan report for 192.168.1.128
Host is up.
Nmap scan report for 192-168-1-141 (192.168.1.141)
Host is up.
Nmap done: 256 IP addresses (5 hosts up) scanned in 2.29 seconds
```

### nslookup

> 检查DNS解析

```bash
apt install dnsutils -y
```

```bash
root@192-168-1-141:~# nslookup www.tanmer.com 8.8.8.8
Server:		8.8.8.8
Address:	8.8.8.8#53

Non-authoritative answer:
Name:	www.tanmer.com
Address: 120.132.67.49
```

### mail

```bash
apt install mailutils -y
```

### sudo

配置sudo用户不需要密码

```bash
sudo sudovi
```

在打开的文件结尾添加如下内容（注意，一定是文件结尾，否则可能不成功）：

```
ubuntu ALL=(ALL) NOPASSWD:ALL
```

重新登录，测试一下sudo应该不会出现密码提示。


# 安装 VPN服务

## Introduction

Want to access the Internet safely and securely from your smartphone or laptop when connected to an untrusted network such as the WiFi of a hotel or coffee shop? A Virtual Private Network (VPN) allows you to traverse untrusted networks privately and securely as if you were on a private network. The traffic emerges from the VPN server and continues its journey to the destination.

When combined with HTTPS connections, this setup allows you to secure your wireless logins and transactions. You can circumvent geographical restrictions and censorship, and shield your location and any unencrypted HTTP traffic from the untrusted network.

OpenVPN is a full-featured open source Secure Socket Layer (SSL) VPN solution that accommodates a wide range of configurations. In this tutorial, we'll set up an OpenVPN server on a Droplet and then configure access to it from Windows, OS X, iOS and Android. This tutorial will keep the installation and configuration steps as simple as possible for these setups.

## Prerequisites

To complete this tutorial, you will need access to an Ubuntu 16.04 server.

You will need to configure a non-root user with sudo privileges before you start this guide. You can follow our Ubuntu 16.04 initial server setup guide to set up a user with appropriate permissions. The linked tutorial will also set up a firewall, which we will assume is in place during this guide.

When you are ready to begin, log into your Ubuntu server as your sudo user and continue below.

## Step 1: Install OpenVPN

To start off, we will install OpenVPN onto our server. OpenVPN is available in Ubuntu's default repositories, so we can use `apt` for the installation. We will also be installing the `easy-rsa` package, which will help us set up an internal CA (certificate authority) for use with our VPN.

To update your server's package index and install the necessary packages type:

```bash
sudo apt-get update
sudo apt-get install openvpn easy-rsa
```

The needed software is now on the server, ready to be configured.

## Step 2: Set Up the CA Directory

OpenVPN is an TLS/SSL VPN. This means that it utilizes certificates in order to encrypt traffic between the server and clients. In order to issue trusted certificates, we will need to set up our own simple certificate authority (CA).

To begin, we can copy the `easy-rsa` template directory into our home directory with the `make-cadir` command:

```bash
make-cadir ~/openvpn-ca
```

Move into the newly created directory to begin configuring the CA:

```bash
cd ~/openvpn-ca
```

## Step 3: Configure the CA Variables

To configure the values our CA will use, we need to edit the vars file within the directory. Open that file now in your text editor:

```bash
vi vars
```

Inside, you will find some variables that can be adjusted to determine how your certificates will be created. We only need to worry about a few of these.

Towards the bottom of the file, find the settings that set field defaults for new certificates. It should look something like this:

{% code title="\~/openvpn-ca/vars" %}

```bash
. . .

export KEY_COUNTRY="US"
export KEY_PROVINCE="CA"
export KEY_CITY="SanFrancisco"
export KEY_ORG="Fort-Funston"
export KEY_EMAIL="me@myhost.mydomain"
export KEY_OU="MyOrganizationalUnit"

. . .
```

{% endcode %}

Edit the values in red to whatever you'd prefer, but do not leave them blank:

{% code title="\~/openvpn-ca/vars" %}

```bash
. . .

export KEY_COUNTRY="US"
export KEY_PROVINCE="NY"
export KEY_CITY="New York City"
export KEY_ORG="DigitalOcean"
export KEY_EMAIL="admin@example.com"
export KEY_OU="Community"

. . .
```

{% endcode %}

While we are here, we will also edit the `KEY_NAME` value just below this section, which populates the subject field. To keep this simple, we'll call it `server` in this guide:

{% code title="\~/openvpn-ca/vars" %}

```bash
export KEY_NAME="server"
```

{% endcode %}

When you are finished, save and close the file.

## Step 4: Build the Certificate Authority

Now, we can use the variables we set and the `easy-rsa` utilities to build our certificate authority.

Ensure you are in your CA directory, and then source the `vars` file you just edited:

```bash
cd ~/openvpn-ca
source vars
```

You should see the following if it was sourced correctly:

```
NOTE: If you run ./clean-all, I will be doing a rm -rf on /home/sammy/openvpn-ca/keys
```

Make sure we're operating in a clean environment by typing:

```bash
./clean-all
```

Now, we can build our root CA by typing:

```bash
./build-ca
```

This will initiate the process of creating the root certificate authority key and certificate. Since we filled out the vars file, all of the values should be populated automatically. Just press `ENTER` through the prompts to confirm the selections:

```
Generating a 2048 bit RSA private key
..........................................................................................+++
...............................+++
writing new private key to 'ca.key'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [US]:
State or Province Name (full name) [NY]:
Locality Name (eg, city) [New York City]:
Organization Name (eg, company) [DigitalOcean]:
Organizational Unit Name (eg, section) [Community]:
Common Name (eg, your name or your server's hostname) [DigitalOcean CA]:
Name [server]:
Email Address [admin@email.com]:
```

We now have a CA that can be used to create the rest of the files we need.

## Step 5: Create the Server Certificate, Key, and Encryption Files

Next, we will generate our server certificate and key pair, as well as some additional files used during the encryption process.

Start by generating the OpenVPN server certificate and key pair. We can do this by typing:

{% hint style="warning" %}
If you choose a name other than `server` here, you will have to adjust some of the instructions below. For instance, when copying the generated files to the `/etc/openvpn` directroy, you will have to substitute the correct names. You will also have to modify the `/etc/openvpn/server.conf` file later to point to the correct `.crt` and `.key` files.
{% endhint %}

```bash
./build-key-server server
```

Once again, the prompts will have default values based on the argument we just passed in (`server`) and the contents of our `vars` file we sourced.

Feel free to accept the default values by pressing `ENTER`. Do not enter a challenge password for this setup. Towards the end, you will have to enter `y` to two questions to sign and commit the certificate:

```aspnet
. . .

Certificate is to be certified until May  1 17:51:16 2026 GMT (3650 days)
Sign the certificate? [y/n]: y


1 out of 1 certificate requests certified, commit? [y/n]y
Write out database with 1 new entries
Data Base Updated
```

Next, we'll generate a few other items. We can generate a strong Diffie-Hellman keys to use during key exchange by typing:

```bash
./build-dh
```

This might take a few minutes to complete.

Afterwards, we can generate an HMAC signature to strengthen the server's TLS integrity verification capabilities:

```bash
openvpn --genkey --secret keys/ta.key
```

## Step 6: Generate a Client Certificate and Key Pair

Next, we can generate a client certificate and key pair. Although this can be done on the client machine and then signed by the server/CA for security purposes, for this guide we will generate the signed key on the server for the sake of simplicity.

We will generate a single client key/certificate for this guide, but if you have more than one client, you can repeat this process as many times as you'd like. Pass in a unique value to the script for each client.

Because you may come back to this step at a later time, we'll re-source the `vars` file. We will use `client1` as the value for our first certificate/key pair for this guide.

To produce credentials without a password, to aid in automated connections, use the `build-key` command like this:

```bash
cd ~/openvpn-ca
source vars
./build-key-pass client1
```

Again, the defaults should be populated, so you can just hit `ENTER` to continue. Leave the challenge password blank and make sure to enter `y` for the prompts that ask whether to sign and commit the certificate.

## Configure the OpenVPN Service

Next, we can begin configuring the OpenVPN service using the credentials and files we've generated.

### Copy the Files to the OpenVPN Directory

To begin, we need to copy the files we need to the `/etc/openvpn` configuration directory.

We can start with all of the files that we just generated. These were placed within the `~/openvpn-ca/keys` directory as they were created. We need to move our CA cert, our server cert and key, the HMAC signature, and the Diffie-Hellman file:

```bash
cd ~/openvpn-ca/keys
sudo cp ca.crt server.crt server.key ta.key dh2048.pem /etc/openvpn
```

Next, we need to copy and unzip a sample OpenVPN configuration file into configuration directory so that we can use it as a basis for our setup:

```bash
gunzip -c /usr/share/doc/openvpn/examples/sample-config-files/server.conf.gz | sudo tee /etc/openvpn/server.conf
```

### Adjust the OpenVPN Configuration

Now that our files are in place, we can modify the server configuration file:

```bash
sudo vi /etc/openvpn/server.conf
```

Basic Configuration

First, find the HMAC section by looking for the `tls-auth` directive. Remove the ";" to uncomment the `tls-auth` line. Below this, add the `key-direction` parameter set to "0":

{% code title="/etc/openvpn/server.conf" %}

```
tls-auth ta.key 0 # This file is secret
key-direction 0
```

{% endcode %}

Next, find the section on cryptographic ciphers by looking for the commented out `cipher` lines. The `AES-128-CBC` cipher offers a good level of encryption and is well supported. Remove the ";" to uncomment the cipher `AES-128-CBC` line:

{% code title="/etc/openvpn/server.conf" %}

```
cipher AES-128-CBC
```

{% endcode %}

Below this, add an `auth` line to select the HMAC message digest algorithm. For this, `SHA256` is a good choice:

{% code title="/etc/openvpn/server.conf" %}

```
auth SHA256
```

{% endcode %}

Finally, find the `user` and `group` settings and remove the ";" at the beginning of to uncomment those lines:

{% code title="/etc/openvpn/server.conf" %}

```
user nobody
group nogroup
```

{% endcode %}

(Optional) Push DNS Changes to Redirect All Traffic Through the VPN

The settings above will create the VPN connection between the two machines, but will not force any connections to use the tunnel. If you wish to use the VPN to route all of your traffic, you will likely want to push the DNS settings to the client computers.

You can do this, uncomment a few directives that will configure client machines to redirect all web traffic through the VPN. Find the `redirect-gateway` section and remove the semicolon ";" from the beginning of the `redirect-gateway` line to uncomment it:

{% code title="/etc/openvpn/server.conf" %}

```
push "redirect-gateway def1 bypass-dhcp"
```

{% endcode %}

`push "redirect-gateway def1 bypass-dhcp"` 会路由所有流量到VPN服务器，而有时我们只需要访问VPN背后的局域网，那么我们可以改为：

```
push "route 10.9.0.0 255.255.0.0"
```

`10.9.0.0 255.255.0.0`是VPN背后的局域网。

Just below this, find the `dhcp-option` section. Again, remove the ";" from in front of both of the lines to uncomment them:

{% code title="/etc/openvpn/server.conf" %}

```
push "dhcp-option DNS 208.67.222.222"
push "dhcp-option DNS 208.67.220.220"
```

{% endcode %}

This should assist clients in reconfiguring their DNS settings to use the VPN tunnel for as the default gateway.

(Optional) Adjust the Port and Protocol

By default, the OpenVPN server uses port 1194 and the UDP protocol to accept client connections. If you need to use a different port because of restrictive network environments that your clients might be in, you can change the `port` option. If you are not hosting web content your OpenVPN server, port 443 is a popular choice since this is usually allowed through firewall rules.

{% code title="/etc/openvpn/server.conf" %}

```
# Optional!
port 443
```

{% endcode %}

Often if the protocol will be restricted to that port as well. If so, change `proto` from `UDP` to `TCP`:

{% code title="/etc/openvpn/server.conf" %}

```
# Optional!
proto tcp
```

{% endcode %}

(Optional) Point to Non-Default Credentials

If you selected a different name during the `./build-key-server` command earlier, modify the cert and key lines that you see to point to the appropriate `.crt` and `.key` files. If you used the default server, this should already be set correctly:

{% code title="/etc/openvpn/server.conf" %}

```
cert server.crt
key server.key
```

{% endcode %}

When you are finished, save and close the file.

## Step 8: Adjust the Server Networking Configuration

Next, we need to adjust some aspects of the server's networking so that OpenVPN can correctly route traffic.

### Allow IP Forwarding

First, we need to allow the server to forward traffic. This is fairly essential to the functionality we want our VPN server to provide.

We can adjust this setting by modifying the `/etc/sysctl.conf` file:

{% code title="/etc/sysctl.conf" %}

```
net.ipv4.ip_forward=1
```

{% endcode %}

Save and close the file when you are finished.

To read the file and adjust the values for the current session, type:

```bash
sudo sysctl -p
```

### Adjust the UFW Rules to Masquerade Client Connections

If you followed the Ubuntu 16.04 initial server setup guide in the prerequisites, you should have the UFW firewall in place. Regardless of whether you use the firewall to block unwanted traffic (which you almost always should do), we need the firewall in this guide to manipulate some of the traffic coming into the server. We need to modify the rules file to set up masquerading, an `iptables` concept that provides on-the-fly dynamic NAT to correctly route client connections.

Before we open the firewall configuration file to add masquerading, we need to find the public network interface of our machine. To do this, type:

```bash
ip route
```

Your public interface should follow the word "`dev`". For example, this result shows the interface named `eth0`, which is highlighted below:

```
default via 10.9.0.1 dev eth0 onlink
10.9.0.0/16 dev eth0  proto kernel  scope link  src 10.9.50.143
```

When you have the interface associated with your default route, open the `/etc/ufw/before.rules` file to add the relevant configuration:

```bash
sudo vi /etc/ufw/before.rules
```

This file handles configuration that should be put into place before the conventional UFW rules are loaded. Towards the top of the file, add the highlighted lines below. This will set the default policy for the `POSTROUTING` chain in the `nat` table and masquerade any traffic coming from the VPN:

{% hint style="warning" %}
Remember to replace `eth0` in the `-A POSTROUTING` line below with the interface you found in the above command.
{% endhint %}

{% code title="/etc/ufw/before.rules" %}

```
#
# rules.before
#
# Rules that should be run before the ufw command line added rules. Custom
# rules should be added to one of these chains:
#   ufw-before-input
#   ufw-before-output
#   ufw-before-forward
#

# START OPENVPN RULES
# NAT table rules
*nat
:POSTROUTING ACCEPT [0:0] 
# Allow traffic from OpenVPN client to eth0 (change to the interface you discovered!)
-A POSTROUTING -s 10.8.0.0/8 -o eth0 -j MASQUERADE
COMMIT
# END OPENVPN RULES

# Don't delete these required lines, otherwise there will be errors
*filter
. . .
```

{% endcode %}

Save and close the file when you are finished.

{% hint style="info" %}
注意：这里的IP`10.8.0.0/8`是VPN分配的地址范围，对应配置在`/etc/openvpn/server.conf`中的`server 10.8.0.0 255.255.0.0`
{% endhint %}

We need to tell UFW to allow forwarded packets by default as well. To do this, we will open the `/etc/default/ufw` file:

```bash
sudo vi /etc/default/ufw
```

Inside, find the `DEFAULT_FORWARD_POLICY` directive. We will change the value from `DROP` to `ACCEPT`:

{% code title="/etc/default/ufw" %}

```
DEFAULT_FORWARD_POLICY="ACCEPT"
```

{% endcode %}

Save and close the file when you are finished.

### Open the OpenVPN Port and Enable the Changes

Next, we'll adjust the firewall itself to allow traffic to OpenVPN.

If you did not change the port and protocol in the `/etc/openvpn/server.conf` file, you will need to open up `UDP` traffic to port `1194`. If you modified the port and/or protocol, substitute the values you selected here.

We'll also add the SSH port in case you forgot to add it when following the prerequisite tutorial:

```bash
sudo ufw allow 1194/udp
sudo ufw allow OpenSSH
```

Now, we can disable and re-enable UFW to load the changes from all of the files we've modified:

```bash
sudo ufw disable
sudo ufw enable
```

Our server is now configured to correctly handle OpenVPN traffic.

## Step 9: Start and Enable the OpenVPN Service

We're finally ready to start the OpenVPN service on our server. We can do this using systemd.

We need to start the OpenVPN server by specifying our configuration file name as an instance variable after the systemd unit file name. Our configuration file for our server is called `/etc/openvpn/server.conf`, so we will add `@server` to end of our unit file when calling it:

```bash
sudo systemctl start openvpn@server
```

Double-check that the service has started successfully by typing:

```bash
sudo systemctl status openvpn@server
```

If everything went well, your output should look something that looks like this:

```
openvpn@server.service - OpenVPN connection to server
   Loaded: loaded (/lib/systemd/system/openvpn@.service; disabled; vendor preset: enabled)
   Active: active (running) since Tue 2016-05-03 15:30:05 EDT; 47s ago
     Docs: man:openvpn(8)
           https://community.openvpn.net/openvpn/wiki/Openvpn23ManPage
           https://community.openvpn.net/openvpn/wiki/HOWTO
  Process: 5852 ExecStart=/usr/sbin/openvpn --daemon ovpn-%i --status /run/openvpn/%i.status 10 --cd /etc/openvpn --script-security 2 --config /etc/openvpn/%i.conf --writepid /run/openvpn/%i.pid (code=exited, sta
 Main PID: 5856 (openvpn)
    Tasks: 1 (limit: 512)
   CGroup: /system.slice/system-openvpn.slice/openvpn@server.service
           └─5856 /usr/sbin/openvpn --daemon ovpn-server --status /run/openvpn/server.status 10 --cd /etc/openvpn --script-security 2 --config /etc/openvpn/server.conf --writepid /run/openvpn/server.pid

May 03 15:30:05 openvpn2 ovpn-server[5856]: /sbin/ip addr add dev tun0 local 10.8.0.1 peer 10.8.0.2
May 03 15:30:05 openvpn2 ovpn-server[5856]: /sbin/ip route add 10.8.0.0/24 via 10.8.0.2
May 03 15:30:05 openvpn2 ovpn-server[5856]: GID set to nogroup
May 03 15:30:05 openvpn2 ovpn-server[5856]: UID set to nobody
May 03 15:30:05 openvpn2 ovpn-server[5856]: UDPv4 link local (bound): [undef]
May 03 15:30:05 openvpn2 ovpn-server[5856]: UDPv4 link remote: [undef]
May 03 15:30:05 openvpn2 ovpn-server[5856]: MULTI: multi_init called, r=256 v=256
May 03 15:30:05 openvpn2 ovpn-server[5856]: IFCONFIG POOL: base=10.8.0.4 size=62, ipv6=0
May 03 15:30:05 openvpn2 ovpn-server[5856]: IFCONFIG POOL LIST
May 03 15:30:05 openvpn2 ovpn-server[5856]: Initialization Sequence Completed
```

You can also check that the OpenVPN `tun0` interface is available by typing:

```bash
ip addr show tun0
```

You should see a configured interface:

```
4: tun0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 100
    link/none 
    inet 10.8.0.1 peer 10.8.0.2/32 scope global tun0
       valid_lft forever preferred_lft forever
```

If everything went well, enable the service so that it starts automatically at boot:

```bash
sudo systemctl enable openvpn@server
```

## Step 10: Create Client Configuration Infrastructure

Next, we need to set up a system that will allow us to create client configuration files easily.

### Creating the Client Config Directory Structure

Create a directory structure within your home directory to store the files:

```bash
mkdir -p ~/client-configs/files
```

Since our client configuration files will have the client keys embedded, we should lock down permissions on our inner directory:

```bash
chmod 700 ~/client-configs/files
```

### Creating a Base Configuration

Next, let's copy an example client configuration into our directory to use as our base configuration:

```bash
cp /usr/share/doc/openvpn/examples/sample-config-files/client.conf ~/client-configs/base.conf
```

Open this new file in your text editor:

```bash
vi ~/client-configs/base.conf
```

Inside, we need to make a few adjustments.

First, locate the `remote` directive. This points the client to our OpenVPN server address. This should be the public IP address of your OpenVPN server. If you changed the port that the OpenVPN server is listening on, change `1194` to the port you selected:

{% code title="\~/client-configs/base.conf" %}

```
. . .
# The hostname/IP and port of the server.
# You can have multiple remote entries
# to load balance between the servers.
remote server_IP_address 1194
. . .
```

{% endcode %}

Be sure that the protocol matches the value you are using in the server configuration:

{% code title="\~/client-configs/base.conf" %}

```
proto udp
```

{% endcode %}

Next, uncomment the `user` and `group` directives by removing the ";":

{% code title="\~/client-configs/base.conf" %}

```
# Downgrade privileges after initialization (non-Windows only)
user nobody
group nogroup
```

{% endcode %}

Find the directives that set the `ca`, `cert`, and `key`. Comment out these directives since we will be adding the certs and keys within the file itself:

{% code title="\~/client-configs/base.conf" %}

```
# SSL/TLS parms.
# See the server config file for more
# description.  It's best to use
# a separate .crt/.key file pair
# for each client.  A single ca
# file can be used for all clients.
#ca ca.crt
#cert client.crt
#key client.key
```

{% endcode %}

Mirror the `cipher` and `auth` settings that we set in the `/etc/openvpn/server.conf` file:

{% code title="\~/client-configs/base.conf" %}

```
cipher AES-128-CBC
auth SHA256
```

{% endcode %}

Next, add the `key-direction` directive somewhere in the file. This must be set to "1" to work with the server:

{% code title="\~/client-configs/base.conf" %}

```
key-direction 1
```

{% endcode %}

Finally, add a few commented out lines. We want to include these with every config, but should only enable them for Linux clients that ship with a `/etc/openvpn/update-resolv-conf` file. This script uses the resolvconf utility to update DNS information for Linux clients.

{% code title="\~/client-configs/base.conf" %}

```
# script-security 2
# up /etc/openvpn/update-resolv-conf
# down /etc/openvpn/update-resolv-conf
```

{% endcode %}

If your client is running Linux and has an `/etc/openvpn/update-resolv-conf` file, you should uncomment these lines from the generated OpenVPN client configuration file.

Save the file when you are finished.

### Creating a Configuration Generation Script

Next, we will create a simple script to compile our base configuration with the relevant certificate, key, and encryption files. This will place the generated configuration in the `~/client-configs/files` directory.

Create and open a file called `make_config.sh` within the `~/client-configs` directory:

```bash
vi ~/client-configs/make_config.sh
```

Inside, paste the following script:

{% code title="\~/client-configs/make\_config.sh" %}

```
#!/bin/bash

# First argument: Client identifier

KEY_DIR=~/openvpn-ca/keys
OUTPUT_DIR=~/client-configs/files
BASE_CONFIG=~/client-configs/base.conf

cat ${BASE_CONFIG} \
    <(echo -e '<ca>') \
    ${KEY_DIR}/ca.crt \
    <(echo -e '</ca>\n<cert>') \
    ${KEY_DIR}/${1}.crt \
    <(echo -e '</cert>\n<key>') \
    ${KEY_DIR}/${1}.key \
    <(echo -e '</key>\n<tls-auth>') \
    ${KEY_DIR}/ta.key \
    <(echo -e '</tls-auth>') \
    > ${OUTPUT_DIR}/${1}.ovpn
```

{% endcode %}

Save and close the file when you are finished.

Mark the file as executable by typing:

```bash
chmod 700 ~/client-configs/make_config.sh
```

## Step 11: Generate Client Configurations

Now, we can easily generate client configuration files.

If you followed along with the guide, you created a client certificate and key called `client1.crt` and `client1.key` respectively by running the `./build-key client1` command in step 6. We can generate a config for these credentials by moving into our `~/client-configs` directory and using the script we made:

```bash
cd ~/client-configs
./make_config.sh client1
```

If everything went well, we should have a `client1.ovpn` file in our `~/client-configs/files` directory:

```bash
ls ~/client-configs/files
```

{% code title="Output" %}

```
client1.ovpn
```

{% endcode %}

### Transferring Configuration to Client Devices

We need to transfer the client configuration file to the relevant device. For instance, this could be your local computer or a mobile device.

While the exact applications used to accomplish this transfer will depend on your choice and device's operating system, you want the application to use SFTP (SSH file transfer protocol) or SCP (Secure Copy) on the backend. This will transport your client's VPN authentication files over an encrypted connection.

Here is an example SFTP command using our client1.ovpn example. This command can be run from your local computer (OS X or Linux). It places the .ovpn file in your home directory:

### Step 12: Install the Client Configuration <a href="#step-12-install-the-client-configuration" id="step-12-install-the-client-configuration"></a>

Now, we'll discuss how to install a client VPN profile on Windows, OS X, iOS, and Android. None of these client instructions are dependent on one another, so feel free to skip to whichever is applicable to you.

The OpenVPN connection will be called whatever you named the .ovpn file. In our example, this means that the connection will be called client1.ovpn for the first client file we generated.

### Windows

#### Installing

The OpenVPN client application for Windows can be found on[ OpenVPN's Downloads ](https://openvpn.net/index.php/open-source/downloads.html)page. Choose the appropriate installer version for your version of Windows.

### OS X

#### Installing

[Tunnelblick](https://tunnelblick.net/) is a free, open source OpenVPN client for Mac OS X. You can download the latest disk image from the [Tunnelblick Downloads](https://tunnelblick.net/downloads.html) page. Double-click the downloaded .dmg file and follow the prompts to install.

Towards the end of the installation process, Tunnelblick will ask if you have any configuration files. It can be easier to answer No and let Tunnelblick finish. Open a Finder window and double-click client1.ovpn. Tunnelblick will install the client profile. Administrative privileges are required.

### Step 13: Test Your VPN Connection <a href="#step-13-test-your-vpn-connection" id="step-13-test-your-vpn-connection"></a>

Once everything is installed, a simple check confirms everything is working properly. Without having a VPN connection enabled, open a browser and go to [DNSLeakTest](https://www.dnsleaktest.com/).

The site will return the IP address assigned by your internet service provider and as you appear to the rest of the world. To check your DNS settings through the same website, click on Extended Test and it will tell you which DNS servers you are using.

Now connect the OpenVPN client to your Droplet's VPN and refresh the browser. The completely different IP address of your VPN server should now appear. That is now how you appear to the world. Again, DNSLeakTest's Extended Test will check your DNS settings and confirm you are now using the DNS resolvers pushed by your VPN.

### Step 14: Revoking Client Certificates <a href="#step-14-revoking-client-certificates" id="step-14-revoking-client-certificates"></a>

Occasionally, you may need to revoke a client certificate to prevent further access to the OpenVPN server.

To do so, enter your CA directory and `re-source` the vars file:

```bash
cd ~/openvpn-ca
source vars
```

Next, call the `revoke-full` command using the client name that you wish to revoke:

```bash
./revoke-full client3
```

This will show some output, ending in `error 23`. This is normal and the process should have successfully generated the necessary revocation information, which is stored in a file called `crl.pem` within the `keys` subdirectory.

Transfer this file to the `/etc/openvpn` configuration directory:

```bash
sudo cp ~/openvpn-ca/keys/crl.pem /etc/openvpn
```

Next, open the OpenVPN server configuration file:

```
sudo nano /etc/openvpn/server.conf
```

At the bottom of the file, add the `crl-verify` option, so that the OpenVPN server checks the certificate revocation list that we've created each time a connection attempt is made:

{% code title="/etc/openvpn/server.conf" %}

```
crl-verify crl.pem
```

{% endcode %}

Save and close the file.

Finally, restart OpenVPN to implement the certificate revocation:

```bash
sudo systemctl restart openvpn@server
```

The client should now longer be able to successfully connect to the server using the old credential.

To revoke additional clients, follow this process:

1. Generate a new certificate revocation list by sourcing the vars file in the `~/openvpn-ca` directory and then calling the `revoke-full` script on the client name.
2. Copy the new certificate revocation list to the `/etc/openvpn` directory to overwrite the old list.
3. Restart the OpenVPN service.

This process can be used to revoke any certificates that you've previously issued for your server.


# 安装DNSMasq

## 安装

```bash
sudo apt update
sudo apt install dnsmasq
```

## 配置

默认配置，外部电脑是访问DNS `53` 端口的，需要配置`listen-address`

查看本机IP地址：

```
ubuntu@10-9-124-19:~$ ip addr
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1454 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 52:54:00:c9:32:c3 brd ff:ff:ff:ff:ff:ff
    inet 10.9.124.19/16 brd 10.9.255.255 scope global eth0
       valid_lft forever preferred_lft forever
```

然后修改`/etc/dnsmasq.conf`

{% code title="/etc/dnsmasq.conf" %}

```
listen-address=127.0.0.1,10.9.124.19
```

{% endcode %}

重启服务

```
sudo systemctl restart dnsmasq
```

## 测试DNS解析

在网内另一台机器执行nslookup检查DNS解析是否可用

```
➜  ~ nslookup www.qq.com 10.9.124.19
Server:		10.9.124.19
Address:	10.9.124.19#53

Non-authoritative answer:
Name:	www.qq.com
Address: 180.163.26.39
```

## 简单解决墙内DNS污染

<https://doc.tanmer.cn/ubuntu#jin-jie-shi>


# Keepalived

Keepalived 结合VIP，可以帮助我们实现IP自动漂移，当一台主机下线，IP会漂移到另外一台主机上，实现IP上的服务高可用

## 安装

```bash
apt update
apt install keepalived -y
```

## 配置

下面配置的`10.9.181.169`是VIP地址

{% code title="/etc/keepalived/keepalived.conf" %}

```
global_defs {
  vrrp_version 3
  vrrp_iptables KEEPALIVED-VIP
}

vrrp_instance vips {
  state BACKUP
  interface eth0
  virtual_router_id 50
  priority 100
  nopreempt
  advert_int 1

  track_interface {
    eth0
  }

  virtual_ipaddress {
    10.9.181.169
  }
}
```

{% endcode %}

## 服务生效

```bash
systemctl restart keepalived
```

{% hint style="info" %}
注意：每台需要IP漂移到的目标主机，都要执行以上安装和配置步骤。
{% endhint %}

## 排错

在云主机上测试，很可能Keepalived不会自动漂移VIP，查看日志发现每天主机都变成了Master

```
RRP_Instance(vips) Transition to MASTER STATE
VRRP_Instance(vips) Entering MASTER STATE
```

这很可能是因为网络交换机过滤了`multicast`数据，参考 <https://serverfault.com/questions/512153/both-servers-running-keepalived-become-master-and-have-a-same-virtual-ip>

如果无法关闭多播的过滤，那么可以用`unicast_peer`指定广播到Keepalived主机的IP地址列表

```
vrrp_instance VI_1 {
  state MASTER
  interface eth0
  #unicast peer 格式必须完全匹配！否则会起不来，必须写成三行。
  unicast_peer {
    192.168.0.10
    192.168.0.11
    192.168.0.12
  }
  ...
```


# OpenSSL 使用技巧

## 查看证书信息

```bash
openssl x509 -in ca.crt -noout -text
```

## 查看在线域名的证书信息

```bash
openssl s_client -connect ms.icometrix.com:443 -cipher 'DEFAULT:!ECDH'
```


# Git

## Git flow开发流程

### 安装Git flow

```bash
curl -L https://raw.githubusercontent.com/nvie/gitflow/develop/contrib/gitflow-installer.sh -o /tmp/gitflow-installer.sh
sudo bash /tmp/gitflow-installer.sh
```

### 初始化

开始规定工作流，进入`develop`分支开发

```bash
git flow init -d
```

### 开始一个Feature

```bash
git flow feature start <your feature>
```

完成一个Feature

```bash
git flow feature finish <your feature>
```

上传一个 Feature

```bash
git flow feature publish <name>
```

或上传一个Feature到某个remote

```bash
git flow feature pull <remote> <name>
```

### 开始一个Release分支

```bash
git flow release
git flow release start <release> [<base>]
git flow release finish <release>
```

### 开始一个Hotfix分支

```bash
git flow hotfix
git flow hotfix start <release> [<base>]
git flow hotfix finish <release>
```


# Nginx


# 自动更新SSL证书

## 第一步，安装certbot

```bash
sudo add-apt-repository ppa:certbot/certbot
sudo apt-get update
sudo apt-get install python-certbot-nginx
```

## 第二步，生成证书

```bash
sudo certbot --nginx -d example.com -d www.example.com
```

{% hint style="warning" %}
注意：此命令会自动搜索出对应的nginx虚拟主机配置文件，然后替换SSL相关的证书地址，执行命令之前，建议先备份配置文件（如：/etc/nginx/site-enabled/example.com）

端口必须是`80` Let's encrpyt不支持80/443端口以外的验证。如果是其他端口，唯一办法就是修改用DNS TXT解析的方法实现。这个不在Certbot工具的功能范围呢。
{% endhint %}

如果是第一次运行 `certbot`，会提示我们输入email并同意使用协议，这里email建议使用真实地址，方便接收证书过期提示。

## 第三步，定期更新证书

用apt-get install certbot之后，系统会安装一个服务和一个定时器，每天certbot会启动两次，会提前1个月为域名续期。

```bash
root@10-10-21-131:~# systemctl status certbot.service
● certbot.service - Certbot
   Loaded: loaded (/lib/systemd/system/certbot.service; static; vendor preset: enabled)
   Active: inactive (dead)
     Docs: file:///usr/share/doc/python-certbot-doc/html/index.html
           https://letsencrypt.readthedocs.io/en/latest/
root@10-10-21-131:~# systemctl status certbot.timer
● certbot.timer - Run certbot twice daily
   Loaded: loaded (/lib/systemd/system/certbot.timer; enabled; vendor preset: enabled)
   Active: active (waiting) since Wed 2018-10-17 10:57:51 CST; 31min ago
```

## 结论

在Ubuntu上安装的certbot服务，会自动更新证书，一次配置，以后再也不用担心证书过期问题了。操作非常简单，5分钟之内即可完成上面操作。

参考：

<https://www.digitalocean.com/community/tutorials/how-to-secure-nginx-with-let-s-encrypt-on-ubuntu-16-04>


# 使用stream模块实现负载均衡

修改nginx配置文件，从`stream-enabled`目录中读取负载均衡的配置文件

```bash
sudo vi /etc/nginx/nginx.conf
```

{% code title="/etc/nginx/nginx.conf" %}

```
stream {
        include /etc/nginx/stream-enabled/*;
}
```

{% endcode %}

```bash
sudo mkdir /etc/nginx/stream-enabled/
sudo mkdir /etc/nginx/stream-available/
```

创建一个负载均衡配置文件：

```bash
sudo vi /etc/nginx/stream-available/kubernetes
```

```
upstream kubernetes {
	server 10.100.0.117:6443;
	server 10.100.0.118:6443;
}

server {
	listen 6443;
	proxy_pass kubernetes;
}
```

检查端口`6443`是否开始监听

```
$ netstat -nptl

roto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      891/sshd
tcp        0      0 0.0.0.0:6443            0.0.0.0:*               LISTEN      27342/nginx -g daem
```

检查数据是否转发过来：

```
$ curl -k https://localhost:6443
{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {

  },
  "status": "Failure",
  "message": "forbidden: User \"system:anonymous\" cannot get path \"/\"",
  "reason": "Forbidden",
  "details": {

  },
  "code": 403
}
```


# 机器学习

## **PredictionIO**

ProdictionIO 是Apache开源的机器学习服务，可以快速搭建支持REST的集数据训练、数据预测的一套系统。

Docker安装方式：<https://github.com/mingfang/docker-predictionio>

Demo：<http://predictionio.apache.org/demo/tapster/>

Ruby SDK: <https://github.com/apache/predictionio-sdk-ruby>

目前官方提供7套算法模板 <https://github.com/apache?utf8=✓&q=predictionio-template&type=&language=>

![](https://3629614825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LAHN5ofIA6hxPSG9Uq3%2F-LFWx3flgPB0x5bOxLmu%2F-LFWyIQTXIEPQs5TykEl%2Fimage.png?alt=media\&token=350072c5-247d-45df-9099-5aeb20fa0ca4)