Docs: Overhaul of Getting started guide (#6811)
* Initial edits to getting started guide * Working on tutorial and getting started guide * Initial edits to getting started guide * Working on tutorial and getting started guide * Continuing work on tutorial * Finished draft of sizing tutorial * Working on getting started guide * Move getting started guide to new file, Netlify redirect * Quick fix to change nodes menu -> my nodes * Finished draft of getting started guide * Fixing broken links from file moving * More tweaks and grammar fixes * Another run through the new pages * Last changes minus screenshot * Fixes to tutorial * Moving things around and final fixes * Addressing Cosmix' comments * Clarified source terminology and added link to modules list * Initial edits to getting started guide * Initial edits to getting started guide * Working on tutorial and getting started guide * Working on tutorial and getting started guide * Continuing work on tutorial * Finished draft of sizing tutorial * Working on getting started guide * Move getting started guide to new file, Netlify redirect * Quick fix to change nodes menu -> my nodes * Finished draft of getting started guide * Fixing broken links from file moving * More tweaks and grammar fixes * Another run through the new pages * Last changes minus screenshot * Fixes to tutorial * Moving things around and final fixes * Addressing Cosmix' comments * Clarified source terminology and added link to modules list * Quick fix to browser line * Fixing one new broken link * One more grammar fix
This commit is contained in:
Joel Hans
Chris Akritidis
parent
352ad4066c
commit
30563fc5ee
|
|
@ -1,182 +0,0 @@
|
|||
# Getting Started
|
||||
|
||||
These are your first steps **after** you have installed Netdata. If you haven't installed it already, please check the [installation page](../packaging/installer).
|
||||
|
||||
## Accessing the dashboard
|
||||
|
||||
To access the Netdata dashboard, navigate with your browser to:
|
||||
|
||||
```
|
||||
http://your.server.ip:19999/
|
||||
```
|
||||
|
||||
<details markdown="1"><summary>Click here, if it does not work.</summary>
|
||||
|
||||
**Verify Netdata is running.**
|
||||
|
||||
Open an ssh session to the server and execute `sudo ps -e | grep netdata`. It should respond with the PID of the Netdata daemon. If it prints nothing, Netdata is not running. Check the [installation page](../packaging/installer) to install it.
|
||||
|
||||
**Verify Netdata responds to HTTP requests.**
|
||||
|
||||
Using the same ssh session, execute `curl -Ss http://localhost:19999`. It should dump on your screen the `index.html` page of the dashboard. If it does not, check the [installation page](../packaging/installer) to install it.
|
||||
|
||||
**Verify Netdata receives the HTTP requests.**
|
||||
|
||||
On the same ssh session, execute `tail -f /var/log/netdata/access.log` (if you installed the static 64bit package, use: `tail -f /opt/netdata/var/log/netdata/access.log`). This command will print on your screen all HTTP requests Netdata receives.
|
||||
|
||||
Next, try to access the dashboard using your web browser, using the URL posted above. If nothing is printed on your terminal, the HTTP request is not routed to your Netdata.
|
||||
|
||||
If you are not sure about your server IP, run this for a hint: `ip route get 8.8.8.8 | grep -oP " src [0-9\.]+ "`. It should print the IP of your server.
|
||||
|
||||
If still Netdata does not receive the requests, something is blocking them. A firewall possibly. Please check your network.
|
||||
|
||||
</details> <br/>
|
||||
|
||||
When you install multiple Netdata servers, all your servers will appear at the node menu at the top left of the dashboard. For this to work, you have to manually access just once, the dashboard of each of your Netdata servers.
|
||||
|
||||
The node menu is more than just browser bookmarks. When switching Netdata servers from that menu, any settings of the current view are propagated to the other Netdata server:
|
||||
|
||||
- the current charts panning (drag the charts left or right),
|
||||
- the current charts zooming (`SHIFT` + mouse wheel over a chart),
|
||||
- the highlighted time-frame (`ALT` + select an area on a chart),
|
||||
- the scrolling position of the dashboard,
|
||||
- the theme you use,
|
||||
- etc.
|
||||
|
||||
are all sent over to other Netdata server, to allow you troubleshoot cross-server performance issues easily.
|
||||
|
||||
## Starting and stopping Netdata
|
||||
|
||||
Netdata installer integrates Netdata to your init / systemd environment.
|
||||
|
||||
To start/stop Netdata, depending on your environment, you should use:
|
||||
|
||||
- `systemctl start netdata` and `systemctl stop netdata`
|
||||
- `service netdata start` and `service netdata stop`
|
||||
- `/etc/init.d/netdata start` and `/etc/init.d/netdata stop`
|
||||
|
||||
Once Netdata is installed, the installer configures it to start at boot and stop at shutdown.
|
||||
|
||||
For more information about using these commands, consult your system documentation.
|
||||
|
||||
## Sizing Netdata
|
||||
|
||||
The default installation of Netdata is configured for a small round-robin database: just 1 hour of data. Depending on the memory your system has and the amount you can dedicate to Netdata, you should adapt this. On production systems with limited RAM, we suggest to set this to 3-4 hours. For best results you should set this to 24 or 48 hours.
|
||||
|
||||
For every hour of data, Netdata needs about 25MB of RAM. If you can dedicate about 100MB of RAM to Netdata, you should set its database size to 4 hours.
|
||||
|
||||
To do this, edit `/etc/netdata/netdata.conf` (or `/opt/netdata/etc/netdata/netdata.conf`) and set:
|
||||
|
||||
```
|
||||
[global]
|
||||
history = SECONDS
|
||||
```
|
||||
|
||||
Make sure the `history` line is not commented (comment lines start with `#`).
|
||||
|
||||
1 hour is 3600 seconds, so the number you need to set is the result of `HOURS * 3600`.
|
||||
|
||||
!!! danger
|
||||
Be careful when you set this on production systems. If you set it too high, your system may run out of memory. By default, Netdata is configured to be killed first when the system starves for memory, but better be careful to avoid issues.
|
||||
|
||||
For more information about Netdata memory requirements, [check this page](../database).
|
||||
|
||||
If your kernel supports KSM (most do), you can [enable KSM to half Netdata memory requirement](../database#ksm).
|
||||
|
||||
## Service discovery and auto-detection
|
||||
|
||||
Netdata supports auto-detection of data collection sources. It auto-detects almost everything: database servers, web servers, dns server, etc.
|
||||
|
||||
This auto-detection process happens **only once**, when Netdata starts. To have Netdata re-discover data sources, you need to restart it. There are a few exceptions to this:
|
||||
|
||||
- containers and VMs are auto-detected forever (when Netdata is running at the host).
|
||||
- many data sources are collected but are silenced by default, until there is useful information to collect (for example network interface dropped packet, will appear after a packet has been dropped).
|
||||
- services that are not optimal to collect on all systems, are disabled by default.
|
||||
- services we received feedback from users that caused issues when monitored, are also disabled by default (for example, `chrony` is disabled by default, because CentOS ships a version of it that uses 100% CPU when queried for statistics).
|
||||
|
||||
Once a data collection source is detected, Netdata will never quit trying to collect data from it, until Netdata is restarted. So, if you stop your web server, Netdata will pick it up automatically when it is started again.
|
||||
|
||||
Since Netdata is installed on all your systems (even inside containers), auto-detection is limited to `localhost`. This simplifies significantly the security model of a Netdata monitored infrastructure, since most applications allow `localhost` access by default.
|
||||
|
||||
A few well known data collection sources that commonly need to be configured are:
|
||||
|
||||
- [systemd services utilization](../collectors/cgroups.plugin/#monitoring-systemd-services) are not exposed by default on most systems, so `systemd` has to be configured to expose those metrics.
|
||||
|
||||
## Configuration quick start
|
||||
|
||||
In Netdata we have:
|
||||
|
||||
- **internal** data collection plugins (running inside the Netdata daemon)
|
||||
- **external** data collection plugins (independent processes, sending data to Netdata over pipes)
|
||||
- modular plugin **orchestrators** (external plugins that have multiple data collection modules)
|
||||
|
||||
You can enable and disable plugins (internal and external) via `netdata.conf` at the section `[plugins]`.
|
||||
|
||||
All plugins have dedicated sections in `netdata.conf`, like `[plugin:XXX]` for overwriting their default data collection frequency and providing additional command line options to them.
|
||||
|
||||
All external plugins have their own `.conf` file.
|
||||
|
||||
All modular plugin orchestrators have a directory in `/etc/netdata` with a `.conf` file for each of their modules.
|
||||
|
||||
It is complex. So, let's see the whole configuration tree for the `nginx` module of `python.d.plugin`:
|
||||
|
||||
In `netdata.conf` at the `[plugins]` section, `python.d.plugin` can be enabled or disabled:
|
||||
|
||||
```
|
||||
[plugins]
|
||||
python.d = yes
|
||||
```
|
||||
|
||||
In `netdata.conf` at the `[plugin:python.d]` section, we can provide additional command line options for `python.d.plugin` and overwite its data collection frequency:
|
||||
|
||||
```
|
||||
[plugin:python.d]
|
||||
update every = 1
|
||||
command options =
|
||||
```
|
||||
|
||||
`python.d.plugin` has its own configuration file for enabling and disabling its modules (here you can disable `nginx` for example):
|
||||
|
||||
```bash
|
||||
sudo /etc/netdata/edit-config python.d.conf
|
||||
```
|
||||
|
||||
Then, `nginx` has its own configuration file for configuring its data collection jobs (most modules can collect data from multiple sources, so the `nginx` module can collect metrics from multiple, local or remote, `nginx` servers):
|
||||
|
||||
```bash
|
||||
sudo /etc/netdata/edit-config python.d/nginx.conf
|
||||
```
|
||||
|
||||
## Health monitoring and alarms
|
||||
|
||||
Netdata ships hundreds of health monitoring alarms for detecting anomalies. These are optimized for production servers.
|
||||
|
||||
Many users install Netdata on workstations and are frustrated by the default alarms shipped with Netdata. On these cases, we suggest to disable health monitoring.
|
||||
|
||||
To disable it, edit `/etc/netdata/netdata.conf` (or `/opt/netdata/etc/netdata/netdata.conf` if you installed the static 64bit package) and set:
|
||||
|
||||
```
|
||||
[health]
|
||||
enabled = no
|
||||
```
|
||||
|
||||
The above will disable health monitoring entirely.
|
||||
|
||||
If you want to keep health monitoring enabled for the dashboard, but you want to disable email notifications, run this:
|
||||
|
||||
```bash
|
||||
sudo /etc/netdata/edit-config health_alarm_notify.conf
|
||||
```
|
||||
|
||||
and set `SEND_EMAIL="NO"`.
|
||||
|
||||
(For static 64bit installations use `sudo /opt/netdata/etc/netdata/edit-config health_alarm_notify.conf`).
|
||||
|
||||
## What is next?
|
||||
|
||||
- Check [Data Collection](../collectors) for configuring data collection plugins.
|
||||
- Check [Health Monitoring](../health) for configuring your own alarms, or setting up alarm notifications.
|
||||
- Check [Streaming](../streaming) for centralizing Netdata metrics.
|
||||
- Check [Backends](../backends) for long term archiving of Netdata metrics to time-series databases.
|
||||
|
||||
[](<>)
|
||||
|
|
@ -107,9 +107,9 @@ folder and either name it `README.md` for generic documentation, or with another
|
|||
|
||||
#### The `docs` folder
|
||||
|
||||
At the root of the Netdata repository is a `docs/` folder. Inside this folder we place documentation that does not
|
||||
have a direct relationship to a specific component of Netdata. It's where we house our [getting started guide](../GettingStarted.md),
|
||||
guides on [running Netdata behind Nginx](../Running-behind-nginx.md), and more.
|
||||
At the root of the Netdata repository is a `docs/` folder. Inside this folder we place documentation that does not have
|
||||
a direct relationship to a specific component of Netdata. It's where we house our [getting started
|
||||
guide](../getting-started.md), guides on [running Netdata behind Nginx](../Running-behind-nginx.md), and more.
|
||||
|
||||
If the documentation you're working on doesn't have a direct relaionship to a component of Netdata,
|
||||
it can be placed in this `docs/` folder.
|
||||
|
|
|
|||
|
|
@ -149,7 +149,7 @@ echo -ne " - 'docs/what-is-netdata.md'
|
|||
- 'packaging/installer/UPDATE.md'
|
||||
- 'packaging/DISTRIBUTIONS.md'
|
||||
- 'packaging/installer/UNINSTALL.md'
|
||||
- 'docs/GettingStarted.md'
|
||||
- 'docs/getting-started.md'
|
||||
- Running Netdata:
|
||||
- 'daemon/README.md'
|
||||
- 'docs/configuration-guide.md'
|
||||
|
|
|
|||
|
|
@ -0,0 +1,233 @@
|
|||
# Getting started guide
|
||||
|
||||
Thanks for trying Netdata! In this guide, we'll quickly walk you through the first steps you should take after getting
|
||||
Netdata installed.
|
||||
|
||||
Netdata can collect thousands of metrics in real-time without any configuration, but there are some valuable things to
|
||||
know to get the most of out Netdata based on your needs.
|
||||
|
||||
> If you haven't installed Netdata yet, visit the [installation instructions](../packaging/installer) for details,
|
||||
> including our one-liner script, which automatically installs Netdata on almost all Linux distributions.
|
||||
|
||||
## Access the dashboard
|
||||
|
||||
Open up your web browser of choice and navigate to `http://YOUR-HOST:19999`. Welcome to Netdata!
|
||||
|
||||
|
||||
|
||||
**What's next?**:
|
||||
|
||||
- Read more about the [standard Netdata dashboard](../web/gui/).
|
||||
- Learn all the specifics of [using charts](../web/README.md#using-charts) or the differences between [charts,
|
||||
context, and families](../web/README.md#charts-contexts-families).
|
||||
|
||||
## Configuration basics
|
||||
|
||||
Netdata primarily uses the `netdata.conf` file for custom configurations.
|
||||
|
||||
On most systems, you can find that file at `/etc/netdata/netdata.conf`.
|
||||
|
||||
> Some operating systems will place your `netdata.conf` at `/opt/netdata/etc/netdata/netdata.conf`, so check there if
|
||||
> you find nothing at `/etc/netdata/netdata.conf`.
|
||||
|
||||
The `netdata.conf` file is broken up into various sections, such as `[global]`, `[web]`, `[registry]`, and more. By
|
||||
default, most options are commented, so you'll have to uncomment them (remove the `#`) for Netdata to recognize your
|
||||
change.
|
||||
|
||||
Once you save your changes, [restart Netdata](#start-stop-and-restart-netdata) to load your new configuration.
|
||||
|
||||
**What's next?**:
|
||||
|
||||
- [Change how long Netdata stores metrics](#change-how-long-netdata-stores-metrics) by either increasing the `history`
|
||||
option or switching to the database engine.
|
||||
- Move Netdata's dashboard to a [different port](https://docs.netdata.cloud/web/server/) or enable TLS/HTTPS
|
||||
encryption.
|
||||
- See all the `netdata.conf` options in our [daemon configuration documentation](../daemon/config/).
|
||||
- Run your own [registry](../registry/README.md#run-your-own-registry).
|
||||
|
||||
## Collect data from more sources
|
||||
|
||||
When Netdata _starts_, it auto-detects dozens of **data sources**, such as database servers, web servers, and more. To
|
||||
auto-detect and collect metrics from a service or application you just installed, you need to [restart
|
||||
Netdata](#start-stop-and-restart-netdata).
|
||||
|
||||
> There is one exception: When Netdata is running on the host (as in not in a container itself), it will always
|
||||
> auto-detect containers and VMs.
|
||||
|
||||
However, auto-detection only works if you installed the source using its standard installation procedure. If Netdata
|
||||
isn't collecting metrics after a restart, your source probably isn't configured correctly. Look at the [external plugin
|
||||
documentation](../collectors/plugins.d/) to find the appropriate module for your source. Those pages will contain more
|
||||
information about how to configure your source for auto-detection.
|
||||
|
||||
Some modules, like `chrony`, are disabled by default and must be enabled manually for auto-detection to work.
|
||||
|
||||
Once Netdata detects a valid source of data, it will continue trying to collect data from it. For example, if
|
||||
Netdata is collecting data from an Nginx web server, and you shut Nginx down, Netdata will collect new data as soon as
|
||||
you start the web server back up—no restart necessary.
|
||||
|
||||
### Configuring plugins
|
||||
|
||||
Even if Netdata auto-detects your service/application, you might want to configure what, or how often, Netdata is
|
||||
collecting data.
|
||||
|
||||
Netdata uses **internal** and **external** plugins to collect data. Internal plugins run within the Netdata dæmon, while
|
||||
external plugins are independent processes that send metrics to Netdata over pipes. There are also plugin
|
||||
**orchestrators**, which are external plugins with one or more data collection **modules**.
|
||||
|
||||
You can configure both internal and external plugins, along with the individual modules. There are many ways to do so:
|
||||
|
||||
- In `netdata.conf`, `[plugins]` section: Enable or disable internal or external plugins with `yes` or `no`.
|
||||
- In `netdata.conf`, `[plugin:XXX]` sections: Each plugin has a section for changing collection frequency or passing
|
||||
options to the plugin.
|
||||
- In `.conf` files for each external plugin: For example, at `/etc/netdata/python.d.conf`.
|
||||
- In `.conf` files for each module : For example, at `/etc/netdata/python.d/nginx.conf`.
|
||||
|
||||
It's complex, so let's walk through an example of the various `.conf` files responsible for collecting data from an
|
||||
Nginx web server using the `nginx` module and the `python.d` plugin orchestrator.
|
||||
|
||||
First, you can enable or disable the `python.d` plugin entirely in `netdata.conf`.
|
||||
|
||||
```conf
|
||||
[plugins]
|
||||
# Enabled
|
||||
python.d = yes
|
||||
# Disabled
|
||||
python.d = no
|
||||
```
|
||||
|
||||
You can also configure the entire `python.d` external plugin via the `[plugin:python.d]` section in `netdata.conf`.
|
||||
Here, you can change how often Netdata uses `python.d` to collect metrics or pass other command options:
|
||||
|
||||
```conf
|
||||
[plugin:python.d]
|
||||
update every = 1
|
||||
command options =
|
||||
```
|
||||
|
||||
The `python.d` plugin has a separate configuration file at `/etc/netdata/python.d.conf` for enabling and disabling
|
||||
modules. You can use the `edit-config` script to edit the file, or open it with your text editor of choice:
|
||||
|
||||
```bash
|
||||
sudo /etc/netdata/edit-config python.d.conf
|
||||
```
|
||||
|
||||
Finally, the `nginx` module has a configuration file called `nginx.conf` in the `python.d` folder. Again, use
|
||||
`edit-config` or your editor of choice:
|
||||
|
||||
```bash
|
||||
sudo /etc/netdata/edit-config python.d/nginx.conf
|
||||
```
|
||||
|
||||
In the `nginx.conf` file, you'll find additional options. The default works in most situations, but you may need to make
|
||||
changes based on your particular Nginx setup.
|
||||
|
||||
**What's next?**:
|
||||
|
||||
- Look at the [full list of data collection modules](Add-more-charts-to-netdata.md#available-data-collection-modules)
|
||||
to configure your sources for auto-detection and monitoring.
|
||||
- Improve the [performance](Performance.md) of Netdata on low-memory systems.
|
||||
- Configure `systemd` to expose [systemd services
|
||||
utilization](../collectors/cgroups.plugin/README.md#monitoring-systemd-services) metrics automatically.
|
||||
- [Reconfigure individual charts](../daemon/config/README.md#per-chart-configuration) in `netdata.conf`.
|
||||
|
||||
## Health monitoring and alarms
|
||||
|
||||
Netdata comes with hundreds of health monitoring alarms for detecting anomalies on production servers. If you're running
|
||||
Netdata on a workstation, you might want to disable Netdata's alarms.
|
||||
|
||||
Edit your `/etc/netdata/netdata.conf` file and set the following:
|
||||
|
||||
```conf
|
||||
[health]
|
||||
enabled = no
|
||||
```
|
||||
|
||||
If you want to keep health monitoring enabled, but turn email notifications off, edit your `health_alarm_notify.conf`
|
||||
file with `edit-config`, or with your the text editor of your choice:
|
||||
|
||||
```bash
|
||||
sudo /etc/netdata/edit-config health_alarm_notify.conf
|
||||
```
|
||||
|
||||
Find the `SEND_EMAIL="YES"` line and change it to `SEND_EMAIL="NO"`.
|
||||
|
||||
**What's next?**:
|
||||
|
||||
- Write your own health alarm using the [examples](../health/README.md#examples).
|
||||
- Add a new notification method, like [Slack](../health/notifications/slack/).
|
||||
|
||||
## Change how long Netdata stores metrics
|
||||
|
||||
By default, Netdata stores 1 hour of historical metrics and uses about 25MB of RAM.
|
||||
|
||||
If that's not enough for you, Netdata is quite adaptable to long-term storage of your system's metrics.
|
||||
|
||||
There are two quick ways to increase the depth of historical metrics: increase the `history` value for the round-robin
|
||||
that's enabled by default, or switch to the database engine.
|
||||
|
||||
We have a tutorial that walks you through both options: [**Changing how long Netdata stores
|
||||
metrics**](tutorials/longer-metrics-storage.md).
|
||||
|
||||
**What's next?**:
|
||||
|
||||
- Learn more about the [memory requirements for the database engine](../database/engine/README.md#memory-requirements)
|
||||
to understand how much RAM/disk space you should commit to storing historical metrics.
|
||||
- Read up on the memory requirements of the [round-robin database](../database/), or figure out whether your system
|
||||
has KSM enabled, which can [reduce the default database's memory usage](../database/README.md#ksm) by about 60%.
|
||||
|
||||
## Monitoring multiple systems with Netdata
|
||||
|
||||
If you have Netdata installed on multiple systems, you can have them all appear in the **My nodes** menu at the top-left
|
||||
corner of the dashboard.
|
||||
|
||||
To show all your servers in that menu, you need to [register for or sign in](netdata-cloud/signing-in.md) to [Netdata
|
||||
Cloud](netdata-cloud/) from each system. Each system will then appear in the **My nodes** menu, which you can use to
|
||||
navigate between your systems quickly.
|
||||
|
||||
|
||||
|
||||
Whenever you pan, zoom, highlight, select, or pause a chart, Netdata will synchronize those settings with any other
|
||||
agent you visit via the My nodes menu. Even your scroll position is synchronized, so you'll see the same charts and
|
||||
respective data for easy comparisons or root cause analysis.
|
||||
|
||||
You can now seamlessly track performance anomalies across your entire infrastructure!
|
||||
|
||||
**What's next?**:
|
||||
|
||||
- Read up on how the [Netdata Cloud registry works](../registry/), and what kind of data it stores and sends to your
|
||||
web browser.
|
||||
- Familiarize yourself with the [Nodes View](netdata-cloud/nodes-view.md)
|
||||
|
||||
## Start, stop, and restart Netdata
|
||||
|
||||
When you install Netdata, it's configured to start at boot, and stop and restart/shutdown. You shouldn't need to start
|
||||
or stop Netdata manually, but you will probably need to restart Netdata at some point.
|
||||
|
||||
- To **start** Netdata, open a terminal and run `service netdata start`.
|
||||
- To **stop** Netdata, run `service netdata stop`.
|
||||
- To **restart** Netdata, run `service netdata restart`.
|
||||
|
||||
The `service` command is a wrapper script that tries to use your system's preferred method of starting or stopping
|
||||
Netdata based on your system. But, if either of those commands fails, try using the equivalent commands for `systemd`
|
||||
and `init.d`:
|
||||
|
||||
- **systemd**: `systemctl start netdata`, `systemctl stop netdata`, `systemctl restart netdata`
|
||||
- **init.d**: `/etc/init.d/netdata start`, `/etc/init.d/netdata stop`, `/etc/init.d/netdata restart`
|
||||
|
||||
## What's next?
|
||||
|
||||
Even after you've configured `netdata.conf`, tweaked alarms, learned the basics of performance troubleshooting, and
|
||||
added all your systems to the **My nodes** menu, you've just gotten started with Netdata.
|
||||
|
||||
Take a look at some more advanced features and configurations:
|
||||
|
||||
- Centralize Netdata metrics from many systems with [streaming](../streaming)
|
||||
- Enable long-term archiving of Netdata metrics via [backends](../backends) to time-series databases.
|
||||
- Improve security by putting Netdata behind an [Nginx proxy with SSL](Running-behind-nginx.md).
|
||||
|
||||
Or, learn more about how you can contribute to [Netdata core](../CONTRIBUTING.md) or our
|
||||
[documentation](contributing/contributing-documentation.md)!
|
||||
|
||||
[](<>)
|
||||
|
|
@ -0,0 +1,155 @@
|
|||
# Change how long Netdata stores metrics
|
||||
|
||||
Netdata helps you collect thousands of system and application metrics every second, but what about storing them for the
|
||||
long term?
|
||||
|
||||
Many people think Netdata can only store about an hour's worth of real-time metrics, but that's just the default
|
||||
configuration today. With the right settings, Netdata is quite capable of efficiently storing hours or days worth of
|
||||
historical, per-second metrics without having to rely on a [backend](../../backends/).
|
||||
|
||||
This tutorial gives two options for configuring Netdata to store more metrics. We recommend the [**database
|
||||
engine**](#using-the-database-engine), as it will soon be the default configuration. However, you can stick with the
|
||||
current default **round-robin database** if you prefer.
|
||||
|
||||
Let's get started.
|
||||
|
||||
## Using the database engine
|
||||
|
||||
The database engine uses RAM to store recent metrics while also using a "spill to disk" feature that takes advantage of
|
||||
available disk space for long-term metrics storage.This feature of the database engine allows you to store a much larger
|
||||
dataset than your system's available RAM.
|
||||
|
||||
The database engine will eventually become the default method of retaining metrics, but until then, you can switch to
|
||||
the database engine by changing a single option.
|
||||
|
||||
Edit your `netdata.conf` file and change the `memory mode` setting to `dbengine`:
|
||||
|
||||
```conf
|
||||
[global]
|
||||
memory mode = dbengine
|
||||
```
|
||||
|
||||
Next, restart Netdata. On Linux systems, we recommend running `sudo service netdata restart`. You're now using the
|
||||
database engine!
|
||||
|
||||
> Learn more about how we implemented the database engine, and our vision for its future, on our blog: [_How and why
|
||||
> we're bringing long-term storage to Netdata_](https://blog.netdata.cloud/posts/db-engine/).
|
||||
|
||||
What makes the database engine efficient? While it's structured like a traditional database, the database engine splits
|
||||
data between RAM and disk. The database engine caches and indexes data on RAM to keep memory usage low, and then
|
||||
compresses older metrics onto disk for long-term storage.
|
||||
|
||||
When the Netdata dashboard queries for historical metrics, the database engine will use its cache, stored in RAM, to
|
||||
return relevant metrics for visualization in charts.
|
||||
|
||||
Now, given that the database engine uses _both_ RAM and disk, there are two other settings to consider: `page cache
|
||||
size` and `dbengine disk space`.
|
||||
|
||||
```conf
|
||||
[global]
|
||||
page cache size = 32
|
||||
dbengine disk space = 256
|
||||
```
|
||||
|
||||
`page cache size` sets the maximum amount of RAM (in MiB) the database engine will use for caching and indexing.
|
||||
`dbengine disk space` sets the maximum disk space (again, in MiB) the database engine will use for storing compressed
|
||||
metrics.
|
||||
|
||||
Based on our testing, these default settings will retain about two day's worth of metrics when Netdata collects 2,000
|
||||
metrics every second.
|
||||
|
||||
If you'd like to change these options, read more about the [database engine's memory
|
||||
footprint](../../database/engine/README.md#memory-requirements).
|
||||
|
||||
With the database engine active, you can back up your `/var/cache/netdata/dbengine/` folder to another location for
|
||||
redundancy.
|
||||
|
||||
Now that you know how to switch to the database engine, let's cover the default round-robin database for those who
|
||||
aren't ready to make the move.
|
||||
|
||||
## Using the round-robin database
|
||||
|
||||
By default, Netdata uses a round-robin database to store 1 hour of per-second metrics. Here's the default setting for
|
||||
`history` in the `netdata.conf` file that comes pre-installed with Netdata.
|
||||
|
||||
```conf
|
||||
[global]
|
||||
history = 3600
|
||||
```
|
||||
|
||||
One hour has 3,600 seconds, hence the `3600` value!
|
||||
|
||||
To increase your historical metrics, you can increase `history` to the number of seconds you'd like to store:
|
||||
|
||||
```conf
|
||||
[global]
|
||||
# 2 hours = 2 * 60 * 60 = 7200 seconds
|
||||
history = 7200
|
||||
# 4 hours = 4 * 60 * 60 = 14440 seconds
|
||||
history = 14440
|
||||
# 24 hours = 24 * 60 * 60 = 86400 seconds
|
||||
history = 86400
|
||||
```
|
||||
|
||||
And so on.
|
||||
|
||||
Next, check to see how many metrics Netdata collects on your system, and how much RAM that uses. Visit the Netdata
|
||||
dashboard and look at the bottom-right corner of the interface. You'll find a sentence similar to the following:
|
||||
|
||||
> Every second, Netdata collects 1,938 metrics, presents them in 299 charts and monitors them with 81 alarms. Netdata is
|
||||
> using 25 MB of memory on **netdata-linux** for 1 hour, 6 minutes and 36 seconds of real-time history.
|
||||
|
||||
On this desktop system, using a Ryzen 5 1600 and 16GB of RAM, the round-robin databases uses 25 MB of RAM to store just
|
||||
over an hour's worth of data for nearly 2,000 metrics.
|
||||
|
||||
To increase the `history` option, you need to edit your `netdata.conf` file and increase the `history` setting. In most
|
||||
installations, you'll find it at `/etc/netdata/netdata.conf`, but some operating systems place it at
|
||||
`/opt/netdata/etc/netdata/netdata.conf`.
|
||||
|
||||
Use `/etc/netdata/edit-config netdata.conf`, or your favorite text editor, to replace `3600` with the number of seconds
|
||||
you'd like to store.
|
||||
|
||||
You should base this number on two things: How much history you need for your use case, and how much RAM you're willing
|
||||
to dedicate to Netdata.
|
||||
|
||||
> Take care when you change the `history` option on production systems. Netdata is configured to stop its process if
|
||||
> your system starts running out of RAM, but you can never be too careful. Out of memory situations are very bad.
|
||||
|
||||
How much RAM will a longer history use? Let's use a little math.
|
||||
|
||||
The round-robin database needs 4 bytes for every value Netdata collects. If Netdata collects metrics every second,
|
||||
that's 4 bytes, per second, per metric.
|
||||
|
||||
```text
|
||||
4 bytes * X seconds * Y metrics = RAM usage in bytes
|
||||
```
|
||||
|
||||
Let's assume your system collects 1,000 metrics per second.
|
||||
|
||||
```text
|
||||
4 bytes * 3600 seconds * 1,000 metrics = 14400000 bytes = 14.4 MB RAM
|
||||
```
|
||||
|
||||
With that formula, you can calculate the RAM usage for much larger history settings.
|
||||
|
||||
```conf
|
||||
# 2 hours at 1,000 metrics per second
|
||||
4 bytes * 7200 seconds * 1,000 metrics = 28800000 bytes = 28.8 MB RAM
|
||||
# 2 hours at 2,000 metrics per second
|
||||
4 bytes * 7200 seconds * 2,000 metrics = 57600000 bytes = 57.6 MB RAM
|
||||
# 4 hours at 2,000 metrics per second
|
||||
4 bytes * 14440 seconds * 2,000 metrics = 115520000 bytes = 115.52 MB RAM
|
||||
# 24 hours at 1,000 metrics per second
|
||||
4 bytes * 86400 seconds * 1,000 metrics = 345600000 bytes = 345.6 MB RAM
|
||||
```
|
||||
|
||||
## What's next?
|
||||
|
||||
Now that you have either configured database engine or round-robin database engine to store more metrics, you'll
|
||||
probably want to see it in action!
|
||||
|
||||
For more information about how to pan charts to view historical metrics, see our documentation on [using
|
||||
charts](../../web/README.md#using-charts).
|
||||
|
||||
And if you'd now like to reduce Netdata's resource usage, view our [performance guide](../Performance.md) for our best
|
||||
practices on optimization.
|
||||
|
|
@ -10,3 +10,7 @@
|
|||
|
||||
# Default build command.
|
||||
command = "./buildhtml.sh"
|
||||
|
||||
[[redirects]]
|
||||
from = "/docs/GettingStarted/"
|
||||
to = "/docs/getting-started"
|
||||
|
|
@ -77,7 +77,7 @@ bash <(curl -Ss https://my-netdata.io/kickstart.sh) --dont-wait --dont-start-it
|
|||
Note: `--stable-channel` and `--local-files` overlap, if you use the tarball override the stable channel option is not effective
|
||||
</details>
|
||||
|
||||
Once Netdata is installed, see [Getting Started](../../docs/GettingStarted.md).
|
||||
Once Netdata is installed, see [Getting Started](../../docs/getting-started.md).
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -149,7 +149,7 @@ sh /tmp/kickstart-static64.sh
|
|||
|
||||
</details>
|
||||
|
||||
Once Netdata is installed, see [Getting Started](../../docs/GettingStarted.md).
|
||||
Once Netdata is installed, see [Getting Started](../../docs/getting-started.md).
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -585,7 +585,7 @@ bash kickstart-static64.sh --local-files /tmp/netdata-version-number-here.gz.run
|
|||
```
|
||||
|
||||
Now that you're finished with your offline installation, you can move on to our
|
||||
[getting started guide](../../docs/GettingStarted.md)!
|
||||
[getting started guide](../../docs/getting-started.md)!
|
||||
|
||||
## Automatic updates
|
||||
|
||||
|
|
|
|||
|
|
@ -43,7 +43,7 @@ analysis.
|
|||
|
||||
Whenever you use a chart in this way, Netdata synchronizes all the other charts
|
||||
to match it. Chart synchronization even works between separate Netdata agents if you connect
|
||||
them using the [node menu](../registry)!
|
||||
them using the [**My nodes** menu](../registry)!
|
||||
|
||||
You can change how charts show their metrics in a few different ways, each of
|
||||
which have a few methods:
|
||||
|
|
|
|||
Loading…
Reference in New Issue