Initial commit
This commit is contained in:
Zhongwei Li
@@ -0,0 +1,16 @@
|
||||
---
|
||||
|
||||
slug: /configurations-overview
|
||||
---
|
||||
|
||||
# Pre-deployment configuration overview
|
||||
|
||||
To ensure that seekdb can be used effectively, you need to configure the operating system and installation environment as follows:
|
||||
|
||||
| Configuration item | Required | Description |
|
||||
| ------ | ------- | ---- |
|
||||
| [Create a user](300.create-regular-user.md) | <ul><li>For personal trial scenarios, you can use the root user.</li><li>For production environments, it is recommended to use a user with sudo privileges.</li></ul> | You can use a regular user with sudo privileges. |
|
||||
| [Plan disks](400.plan-disks.md) | <ul><li>For personal trial scenarios, no configuration is required.</li><li>For production environments, it is recommended to configure.</li></ul> | In production environments, it is recommended to use different disks for the log storage directory and data storage directory to avoid performance issues in high-pressure scenarios when both directories are planned on the same disk. |
|
||||
| [Configure limits.conf](500.configure-limits-conf.md) | <ul><li>For personal trial scenarios, no configuration is required.</li><li>For production environments, configuration is required.</li></ul> | Improves the stability and performance of the cluster during runtime. |
|
||||
| [Configure sysctl.conf](600.configure-sysctl-conf.md) | <ul><li>For personal trial scenarios, no configuration is required.</li><li>For production environments, configuration is required.</li></ul> | Improves Linux system performance, thereby improving seekdb performance. |
|
||||
| [Disable the firewall and SELinux](700.disable-firewall-and-selinux.md) | <ul><li>For personal trial scenarios, no configuration is required.</li><li>For production environments, configuration is required.</li></ul> | Prevents access to the database from being blocked. |
|
||||
@@ -0,0 +1,37 @@
|
||||
---
|
||||
|
||||
slug: /create-regular-user
|
||||
---
|
||||
|
||||
# Create a user
|
||||
|
||||
For enterprise users, when deploying seekdb, it is recommended to use the admin user, or you can use other users. However, the user must have sudo privileges and be the owner of database-related directories. This topic describes how to create a regular user and set sudo privileges for the regular user.
|
||||
|
||||
1. Run the following commands to create the user group admin and create the user admin.
|
||||
|
||||
```shell
|
||||
[root@test001 ~]# useradd -U admin -d /home/admin -s /bin/bash
|
||||
[root@test001 ~]# chown -R admin:admin /home/admin
|
||||
```
|
||||
|
||||
2. Run the following command to set a password for the admin account.
|
||||
|
||||
```shell
|
||||
[root@test001 ~]# passwd admin
|
||||
```
|
||||
|
||||
3. Set sudo privileges for the admin account.
|
||||
|
||||
Run the following command to open the `/etc/sudoers` file:
|
||||
|
||||
```shell
|
||||
[root@test001 ~]# vim /etc/sudoers
|
||||
```
|
||||
|
||||
Add the following content at the end of the `/etc/sudoers` file:
|
||||
|
||||
```shell
|
||||
## Same thing without a password
|
||||
# %wheel ALL=(ALL) NOPASSWD: ALL
|
||||
admin ALL=(ALL) NOPASSWD: ALL
|
||||
```
|
||||
@@ -0,0 +1,269 @@
|
||||
---
|
||||
|
||||
slug: /plan-disks
|
||||
---
|
||||
|
||||
# Plan disks
|
||||
|
||||
seekdb servers depend on data disks, transaction log disks, and seekdb installation disks. If you are a personal user, you can put all data on a single disk and skip this step. If you are an enterprise user, it is recommended to mount data to three separate disks.
|
||||
|
||||
If your machine does not have three disks, or if you are using a RAID disk array, you need to partition the disk or the logical volumes of the disk array. It is recommended that you partition using the following scheme:
|
||||
|
||||
* Data disk
|
||||
|
||||
The data disk is used to store baseline data, and the path is specified by the configuration parameter `data_dir`. When you start seekdb for the first time, `${data_dir}/{sstable,slog}` will be created automatically. The size of the data disk is determined by the `datafile_disk_percentage`/`datafile_size` parameters. You can also dynamically expand disk files after deployment through the `datafile_next` and `datafile_maxsize` configuration items. For details, see [Configure dynamic expansion of disk data files](https://en.oceanbase.com/docs/common-oceanbase-database-10000000001971412).
|
||||
|
||||
* Transaction log disk
|
||||
|
||||
The path of the transaction log disk is specified by the configuration parameter `redo-dir`. It is recommended that you set the size of the transaction log disk to 3 to 4 times or more of the seekdb memory. When you start seekdb for the first time, `${redo-dir}` will be created automatically. The transaction log disk contains multiple fixed-size files, and you can automatically create and clear transaction logs as needed. When transaction logs reach 80% of the total disk capacity, automatic clearing will be triggered. However, transaction logs can only be deleted when the memory data corresponding to the transaction logs has been merged into the baseline data.
|
||||
|
||||
With the same data volume, the size of transaction logs is approximately three times the size of memory data. Therefore, the upper limit of the space required for the transaction log disk is proportional to the total data volume after two merges. Empirical formula: Transaction log file size = 3~4 times the upper limit of incremental data memory.
|
||||
|
||||
* seekdb installation disk
|
||||
|
||||
The path of the seekdb installation disk is specified by the configuration parameter `base-dir`. The seekdb RPM package installation directory is located under `${base-dir}`. Baseline data files and transaction log files are linked to independent data disks and transaction log disks respectively through soft links. seekdb runtime logs are located under `${base-dir}/log`. Runtime logs continue to grow, and seekdb cannot automatically delete runtime logs, so you need to regularly delete runtime logs.
|
||||
|
||||
## Disk mounting
|
||||
|
||||
The disk mount point requirements for seekdb are shown in the following table.
|
||||
|
||||
* Personal users
|
||||
|
||||
For personal users, disk mounting is not required. It is recommended that the minimum available disk space be 5 GB when in use.
|
||||
|
||||
* Enterprise users
|
||||
|
||||
| Directory | Size | Purpose | File system format |
|
||||
|------------|----------------|----------------|-------------------------|
|
||||
| /home | 100 GB~300 GB | seekdb database installation disk | ext4 or xfs recommended |
|
||||
| /data/log1 | 2 times the memory size allocated to seekdb | seekdb process log disk | ext4 or xfs recommended |
|
||||
| /data/1 | Depends on the size of data to be stored | seekdb process data disk | ext4 or xfs recommended |
|
||||
|
||||
:::info
|
||||
<ul><li>It is recommended that the root directory be at least 50 GB. If using LVM, it is recommended to use striping parameters when creating. Example: <code>lvcreate -n data -L 3000G obvg --stripes=3 --stripesize=128</code></li>
|
||||
<li>In production environments, it is recommended to use different disks for the data disk, log disk, and installation disk to avoid performance issues.</li></ul>
|
||||
:::
|
||||
|
||||
## Disk mounting operations
|
||||
|
||||
Disk mounting must be performed under the root user, and there are two operation methods:
|
||||
|
||||
* Mount disks using LVM tools (recommended).
|
||||
* Mount disks using fdisk tools.
|
||||
|
||||
### Mount disks using LVM tools
|
||||
|
||||
1. Check disk information
|
||||
|
||||
Use the `fdisk -l` command to identify available disks and partitions, and confirm the target device (such as `/dev/sdb1`).
|
||||
|
||||
```shell
|
||||
fdisk -l
|
||||
```
|
||||
|
||||
2. Install LVM tools
|
||||
|
||||
If LVM is not pre-installed, run the following command to install LVM. If LVM is already installed, skip this step.
|
||||
|
||||
* Debian/Ubuntu systems
|
||||
|
||||
```shell
|
||||
apt-get install lvm2
|
||||
```
|
||||
|
||||
* CentOS/RHEL systems
|
||||
|
||||
```shell
|
||||
yum install lvm2
|
||||
```
|
||||
|
||||
3. Create a physical volume (PV).
|
||||
|
||||
1. Initialize the partition as a physical volume.
|
||||
|
||||
```shell
|
||||
pvcreate /dev/sdb1
|
||||
```
|
||||
|
||||
2. Verify the PV creation result
|
||||
|
||||
```shell
|
||||
pvs
|
||||
```
|
||||
|
||||
4. Create a volume group (VG).
|
||||
|
||||
1. Combine multiple physical volumes into one VG.
|
||||
|
||||
```shell
|
||||
vgcreate vg01 /dev/sdb1 /dev/sdc1
|
||||
```
|
||||
|
||||
2. View VG information
|
||||
|
||||
```shell
|
||||
vgs
|
||||
```
|
||||
|
||||
5. Create a logical volume (LV).
|
||||
|
||||
1. Create a 100 GB logical volume from the VG.
|
||||
|
||||
```shell
|
||||
lvcreate -L 100G -n lv01 vg01
|
||||
```
|
||||
|
||||
The size of the logical volume here can be set according to actual needs.
|
||||
|
||||
2. View LV information.
|
||||
|
||||
```shell
|
||||
lvs
|
||||
```
|
||||
|
||||
6. Format and mount.
|
||||
|
||||
1. Format as ext4 file system.
|
||||
|
||||
```shell
|
||||
mkfs.ext4 /dev/vg01/lv01
|
||||
```
|
||||
|
||||
2. Create a mount point.
|
||||
|
||||
```shell
|
||||
mkdir -p /data/1
|
||||
```
|
||||
|
||||
3. Temporarily mount.
|
||||
|
||||
```shell
|
||||
mount /dev/vg01/lv01 /data/1
|
||||
```
|
||||
|
||||
7. Set automatic mounting on boot.
|
||||
|
||||
Edit the `/etc/fstab` file and add the mount configuration:
|
||||
|
||||
```shell
|
||||
vim /etc/fstab
|
||||
```
|
||||
|
||||
Add the following content to the configuration file:
|
||||
|
||||
```shell
|
||||
/dev/vg01/lv01 /data/1 ext4 defaults,noatime,nodiratime,nodelalloc,barrier=0 0 0
|
||||
```
|
||||
|
||||
### Mount disks using fdisk tools
|
||||
|
||||
1. Check disk information
|
||||
|
||||
Use the `fdisk -l` command to identify available disks and partitions, and confirm the target device (such as `/dev/sdb1`).
|
||||
|
||||
```shell
|
||||
fdisk -l
|
||||
```
|
||||
|
||||
2. Create a partition
|
||||
|
||||
Use the fdisk tool to create a new partition, for example `fdisk /dev/sdb1`, enter n to create a primary partition, and finally save (w).
|
||||
|
||||
```shell
|
||||
fdisk /dev/sdb1
|
||||
```
|
||||
|
||||
3. Format and mount.
|
||||
|
||||
1. Format as ext4 file system.
|
||||
|
||||
```shell
|
||||
mkfs.ext4 /dev/sdb1
|
||||
```
|
||||
|
||||
2. Create a mount point.
|
||||
|
||||
```shell
|
||||
mkdir -p /data/1
|
||||
```
|
||||
|
||||
3. Temporarily mount.
|
||||
|
||||
```shell
|
||||
mount /dev/sdb1 /data/1
|
||||
```
|
||||
|
||||
4. Set automatic mounting on boot.
|
||||
|
||||
Edit the `/etc/fstab` file and add the mount configuration:
|
||||
|
||||
```shell
|
||||
vim /etc/fstab
|
||||
```
|
||||
|
||||
Add the following content to the configuration file:
|
||||
|
||||
```shell
|
||||
/dev/sdb1 /data/1 ext4 defaults,noatime,nodiratime,nodelalloc,barrier=0 0 0
|
||||
```
|
||||
|
||||
## Check disks
|
||||
|
||||
After disks are mounted, run the following command to check the disk mounting status:
|
||||
|
||||
```shell
|
||||
df -h
|
||||
```
|
||||
|
||||
The following result is returned:
|
||||
|
||||
```shell
|
||||
Filesystem Size Used Avail Use% Mounted on
|
||||
devtmpfs 31G 0 31G 0% /dev
|
||||
tmpfs 31G 0 31G 0% /dev/shm
|
||||
tmpfs 31G 516K 31G 1% /run
|
||||
tmpfs 31G 0 31G 0% /sys/fs/cgroup
|
||||
/dev/vda1 493G 171G 302G 37% /
|
||||
tmpfs 6.2G 0 6.2G 0% /run/user/0
|
||||
/dev/sdb1 984G 77M 934G 1% /data/1
|
||||
/dev/vdc1 196G 61M 186G 1% /data/log1
|
||||
/dev/vdb1 492G 73M 467G 1% /home/admin/seekdb
|
||||
```
|
||||
|
||||
Result description
|
||||
|
||||
* `/data/1` is the data disk with a size of 1 TB.
|
||||
|
||||
* `/data/log1` stores logs.
|
||||
|
||||
* `/home/admin/seekdb` stores seekdb binary files and runtime logs.
|
||||
|
||||
Ensure that the disks corresponding to `data_dir`, `redo_dir`, and `home_path` in the configuration file have been mounted. The directories corresponding to `data_dir` and `redo_dir` are empty, and the disk usage of the directory corresponding to `data_dir` must be less than 4%.
|
||||
|
||||
## Set directory permissions
|
||||
|
||||
After disk mounting is complete, you need to check the permissions of the directories corresponding to the mounted disks.
|
||||
|
||||
Run the following command to check the permissions of cluster-related file directories.
|
||||
|
||||
Here, the `data` directory is used as an example:
|
||||
|
||||
```shell
|
||||
[root@test001 data]# ls -al
|
||||
```
|
||||
|
||||
The following result is returned:
|
||||
|
||||
```shell
|
||||
drwxr-xr-x 2 admin admin 4096 Feb 9 18:43 .
|
||||
drwxr-xr-x 2 admin admin 4096 Feb 9 18:43 log1
|
||||
```
|
||||
|
||||
If you find that the `admin` user does not have permissions for related files after checking directory permissions, run the following command to change the file owner:
|
||||
|
||||
```shell
|
||||
[root@test001 ~]# chown -R admin:admin /data/log1
|
||||
[root@test001 ~]# chown -R admin:admin /data
|
||||
```
|
||||
|
||||
Here, `/data/log1` and `/data` are example mount directories. You need to replace them with your actual mount directories.
|
||||
@@ -0,0 +1,84 @@
|
||||
---
|
||||
|
||||
slug: /configure-limits-conf
|
||||
---
|
||||
|
||||
# Configure limits.conf
|
||||
|
||||
This topic describes how to limit the number of processes by configuring `limits.conf`.
|
||||
|
||||
If you are a personal user, you can skip this step. If you are an enterprise user, it is recommended to configure `limits.conf`.
|
||||
|
||||
You can modify resource limits using the following method:
|
||||
|
||||
Modify at the global level through the configuration file `/etc/security/limits.conf`.
|
||||
|
||||
The limits involved in seekdb processes include the maximum stack space size, maximum number of open files, and core file size.
|
||||
|
||||
## Modify configuration
|
||||
|
||||
Set the maximum stack space size at the session level to `unlimited`, the maximum number of open files to `655350`, and the core file size to `unlimited`.
|
||||
|
||||
Run the following command to open the `/etc/security/limits.conf` configuration file:
|
||||
|
||||
```shell
|
||||
sudo vim /etc/security/limits.conf
|
||||
```
|
||||
|
||||
Add the following content to the `/etc/security/limits.conf` configuration file:
|
||||
|
||||
```shell
|
||||
* soft nofile 655350
|
||||
* hard nofile 655350
|
||||
* soft stack unlimited
|
||||
* hard stack unlimited
|
||||
* soft nproc 655360
|
||||
* hard nproc 655360
|
||||
* soft core unlimited
|
||||
* hard core unlimited
|
||||
```
|
||||
|
||||
:::info
|
||||
|
||||
<ul><li>You need to check whether the nproc configuration exists in the <code>/etc/security/limits.d/20-nproc.conf</code> file. If it exists, you need to synchronously modify the nproc value in that file.</li>
|
||||
<li>The limits.conf configuration only takes effect for newly logged-in sessions (such as newly opened terminals, SSH connections, or newly started processes). Existing sessions or processes will not automatically inherit the new configuration and need to log in again or restart related services.</li></ul>
|
||||
:::
|
||||
|
||||
## View configuration
|
||||
|
||||
Exit the current session and log in again. Run the following command to check whether the configuration has taken effect.
|
||||
|
||||
```shell
|
||||
ulimit -a
|
||||
```
|
||||
|
||||
The output is as follows.
|
||||
|
||||
```shell
|
||||
core file size (blocks, -c) unlimited
|
||||
data seg size (kbytes, -d) unlimited
|
||||
scheduling priority (-e) 0
|
||||
file size (blocks, -f) unlimited
|
||||
pending signals (-i) 252876
|
||||
max locked memory (kbytes, -l) 64
|
||||
max memory size (kbytes, -m) unlimited
|
||||
open files (-n) 655350
|
||||
pipe size (512 bytes, -p) 8
|
||||
POSIX message queues (bytes, -q) 819200
|
||||
real-time priority (-r) 0
|
||||
stack size (kbytes, -s) unlimited
|
||||
cpu time (seconds, -t) unlimited
|
||||
max user processes (-u) 655360
|
||||
virtual memory (kbytes, -v) unlimited
|
||||
file locks (-x) unlimited
|
||||
```
|
||||
|
||||
From the results, you can see:
|
||||
|
||||
* Core file size indicates the maximum threshold of core files (in blocks), corresponding to the core parameter in the `limits.conf` configuration file. Check whether the value is unlimited.
|
||||
|
||||
* Open files indicates the maximum number of open file descriptors, corresponding to the nofile parameter in the `limits.conf` configuration file. Check whether the value is 655350.
|
||||
|
||||
* Stack size indicates the stack size (in kilobytes), corresponding to the stack parameter in the `limits.conf` configuration file. Check whether the value is unlimited.
|
||||
|
||||
* Max user processes indicates the maximum number of user processes, corresponding to the nproc parameter in the `limits.conf` configuration file. Check whether the value is 655360.
|
||||
@@ -0,0 +1,94 @@
|
||||
---
|
||||
|
||||
slug: /configure-sysctl-conf
|
||||
---
|
||||
|
||||
# Configure sysctl.conf
|
||||
|
||||
This topic describes how to improve Linux system performance by modifying the `sysctl.conf` configuration on the machine.
|
||||
|
||||
If you are a personal user, you can skip this step. If you are an enterprise user, it is recommended to configure `sysctl.conf`.
|
||||
|
||||
## Modify configuration
|
||||
|
||||
Run the following command to open the `/etc/sysctl.conf` configuration file:
|
||||
|
||||
```shell
|
||||
sudo vim /etc/sysctl.conf
|
||||
```
|
||||
|
||||
Add the following content to the `/etc/sysctl.conf` configuration file:
|
||||
|
||||
```shell
|
||||
# for seekdb
|
||||
## Modify kernel asynchronous I/O limits
|
||||
fs.aio-max-nr = 1048576
|
||||
|
||||
## Network optimization
|
||||
net.core.somaxconn = 2048
|
||||
net.core.netdev_max_backlog = 10000
|
||||
net.core.rmem_default = 16777216
|
||||
net.core.wmem_default = 16777216
|
||||
net.core.rmem_max = 16777216
|
||||
net.core.wmem_max = 16777216
|
||||
|
||||
net.ipv4.ip_forward = 0
|
||||
net.ipv4.conf.default.rp_filter = 1
|
||||
net.ipv4.conf.default.accept_source_route = 0
|
||||
net.ipv4.tcp_syncookies = 1
|
||||
net.ipv4.tcp_rmem = 4096 87380 16777216
|
||||
net.ipv4.tcp_wmem = 4096 65536 16777216
|
||||
net.ipv4.tcp_max_syn_backlog = 16384
|
||||
net.ipv4.tcp_fin_timeout = 15
|
||||
net.ipv4.tcp_slow_start_after_idle=0
|
||||
|
||||
vm.swappiness = 0
|
||||
vm.min_free_kbytes = 2097152
|
||||
vm.overcommit_memory = 0
|
||||
|
||||
fs.file-max = 6573688
|
||||
fs.pipe-user-pages-soft = 0
|
||||
|
||||
# Modify the number of virtual memory areas a process can own
|
||||
vm.max_map_count = 655360
|
||||
|
||||
# Set the filename format and directory for core files
|
||||
kernel.core_pattern = /data/core-%e-%p-%t
|
||||
```
|
||||
|
||||
Here, `/data` in `kernel.core_pattern` is the `data` directory of seekdb.
|
||||
|
||||
:::tip
|
||||
<ul><li>If the <code>max_map_count</code> configuration is unreasonable, it may cause serious memory leaks.</li>
|
||||
<li>You need to reserve at least the same amount of disk space as the memory_limit configured for seekdb for the core file directory, and avoid affecting the available space of the data directory and log directory.</li></ul>
|
||||
:::
|
||||
|
||||
## Load configuration
|
||||
|
||||
After modifying the configuration, run the following command to load the configuration and make it take effect.
|
||||
|
||||
```shell
|
||||
sysctl -p
|
||||
```
|
||||
|
||||
## ARM environment deployment recommendations
|
||||
|
||||
* Enable NUMA support in BIOS/UEFI and kernel boot parameters.
|
||||
* In ARM and Hygon architecture environments, it is recommended to modify the configuration file `/etc/sysctl.conf` and set the parameters `kernel.numa_balancing`, `vm.zone_reclaim_mode`, and `vm.swappiness` to `0`, as follows:
|
||||
|
||||
```shell
|
||||
[root@xxx /]
|
||||
$vi /etc/sysctl.conf
|
||||
## Disable NUMA balancing to avoid performance jitter during balancing
|
||||
kernel.numa_balancing = 0
|
||||
## Disable memory reclaim and reallocation functions
|
||||
vm.zone_reclaim_mode = 0
|
||||
vm.swappiness = 0
|
||||
```
|
||||
|
||||
Apply the values in the sysctl.conf configuration file:
|
||||
|
||||
```shell
|
||||
[root@xxx /]
|
||||
$sysctl -p
|
||||
```
|
||||
@@ -0,0 +1,44 @@
|
||||
---
|
||||
|
||||
slug: /disable-firewall-and-selinux
|
||||
---
|
||||
|
||||
# Disable the firewall and SELinux
|
||||
|
||||
This topic describes how to disable the firewall and SELinux. If you are a personal user, you can skip this step. If you are an enterprise user, it is recommended that you refer to this topic to disable the firewall and SELinux.
|
||||
|
||||
## Disable the firewall
|
||||
|
||||
Run the following commands in sequence to disable the firewall:
|
||||
|
||||
```shell
|
||||
systemctl disable firewalld
|
||||
systemctl stop firewalld
|
||||
systemctl status firewalld
|
||||
```
|
||||
|
||||
## Disable SELinux
|
||||
|
||||
Run the following command to open the `/etc/selinux/config` configuration file:
|
||||
|
||||
```shell
|
||||
vi /etc/selinux/config
|
||||
```
|
||||
|
||||
Modify the corresponding configuration item in the `/etc/selinux/config` configuration file to the following:
|
||||
|
||||
```shell
|
||||
SELINUX=disabled
|
||||
```
|
||||
|
||||
Run the following command or restart the server to make the changes take effect:
|
||||
|
||||
```shell
|
||||
setenforce 0
|
||||
```
|
||||
|
||||
Run the following command to check whether the changes have taken effect:
|
||||
|
||||
```shell
|
||||
sestatus
|
||||
```
|
||||
Reference in New Issue
Block a user