Commit Graph

220 Commits

Author SHA1 Message Date
Tony Murray a5198d7d57
Remove some lightly used or unused dbFacile functions (#15418)
* Remove some lightly used or unused dbFacile functions
['NULL'] legacy behavior seems unsupported, replace with actual nulls

* Fix refactor mishap

* another

* update baseline

* these were very wrong... what happened?
2023-10-12 09:15:03 -05:00
Tony Murray d865e3b372
Always mark device last_discovered (#15218)
Previously, if the device was ping only, it wasn't marked as discovered.
Now we always run discovery, but basically all it does is update last_discovered.
2023-08-13 16:56:54 +02:00
Tony Murray e4451714e2
version and git helper improvements (#14412)
* Fix up version and git helpers
Improve method names
Move all git calls into the git helper
Allow runtime and external cache of results where appropriate
Consolidate version headers for discovery, poller, and validate

* Style fixes

* improve consistency in git calls

* fix style

* don't send name inconsistently

* Improve database versions

* No need to cache Version it is not used more than once currently.
2022-10-02 00:41:56 -05:00
Jellyfrog 670f964e98
Validate: use "database version" instead of "mysql version" (#14158)
* Validate: use "database version" instead of "mysql version"

* wip

* wip
2022-07-29 11:36:18 -05:00
Tony Murray b6a8b602b8
Oxidized update and Device remove (#13730)
* Oxidized + Device remove
Was just working on oxidized, but then to properly update nodes after delete, updated delete_device()

* revert dumb style changes

* baseline update and no DI there...

* Fix OS first load and device deletion missing tables
2022-01-29 21:09:05 -06:00
Tony Murray c79b187d72
Poller rewrite (Try 2) (#13525)
* core WIP

* try to finish up

* trim space too
and a couple of cleanups

* update test data

* put escapes back

* another net-snmp difference

* correct copy paste error

* WIP

* Use new code YAY

* a tiny bit more

* Kind of working

* Handle manual modules correctly

* convert core to modern module

* Only save metrics if modules is not overridden

* correct module exists check

* database error handling

* debug handling

* restore bad changes

* Introduce Actions
 RunAlertRulesAction
 UpdateDeviceGroupsAction

* tweaks to output

* Fix some issues in outside code

* Style fixes

* fixes to module status checks

* typehints!

* Use logger only and DI

* OS module not named correctly

* Work on quiet output a bit more

* generically don't change output when disabling debug if the driver is already stack

* Fix missing $device variable for legacy os polling
Fix missing dbFacile functions when no legacy modules polled in RunAlertRulesAction

* restore legacy os module shim

* use the new poller code for tests

* PollingDevice event

* Fix some issues and enable/disable error reporting around legacy modules

* typehints

* fully update baseline

* Use Process for version commands so we don't leak debug output.

* don't detect rrdtool version in ci every time

* style fixes

* Warning fixes

* more fixes

* re-update baseline

* remove diff noise

* fix up alerts

* Catch exceptions in device ip lookup

* Revert accidental snmp.inc.php poller target change
(should have been ?: not ??)
2021-11-17 19:23:55 -06:00
Tony Murray f8764ddcb5
Revert "Poller command rewrite (#13414)" (#13524)
This reverts commit 1752d1efd4.
2021-11-16 21:40:29 -06:00
Tony Murray 1752d1efd4
Poller command rewrite (#13414)
* core WIP

* try to finish up

* trim space too
and a couple of cleanups

* update test data

* put escapes back

* another net-snmp difference

* correct copy paste error

* WIP

* Use new code YAY

* a tiny bit more

* Kind of working

* Handle manual modules correctly

* convert core to modern module

* Only save metrics if modules is not overridden

* correct module exists check

* database error handling

* debug handling

* restore bad changes

* Introduce Actions
 RunAlertRulesAction
 UpdateDeviceGroupsAction

* tweaks to output

* Fix some issues in outside code

* Style fixes

* fixes to module status checks

* typehints!

* Use logger only and DI

* OS module not named correctly

* Work on quiet output a bit more

* generically don't change output when disabling debug if the driver is already stack

* Fix missing $device variable for legacy os polling
Fix missing dbFacile functions when no legacy modules polled in RunAlertRulesAction

* restore legacy os module shim

* use the new poller code for tests

* PollingDevice event

* Fix some issues and enable/disable error reporting around legacy modules

* typehints

* fully update baseline

* Use Process for version commands so we don't leak debug output.

* don't detect rrdtool version in ci every time

* style fixes

* Warning fixes

* more fixes

* re-update baseline

* remove diff noise

* fix up alerts
2021-11-16 16:59:46 -06:00
Tony Murray 8d5bc3fc41
Use Measurements for all statistic collection (#13333)
* Use Measurements for all statistic collection

* fix space

* Use colors
cleanup

* fix style

* manually fix license notice

* add return times and new line

* add return times and new line

* fix mistake in copyright template

* fix style
2021-10-06 17:09:54 -05:00
Tony Murray d496c228d4
Discovery: mark snmp disabled devices as skipped (#13202) 2021-09-08 18:15:08 +02:00
Tony Murray 61c89794e4
Remove debug globals (#12811)
* Remove $debug global
and $vdebug global
makes these variables more accessible and protects from collisions.

* the on boot set sends application as the first parameter, just handle that

* Relocate other debug related functions

* Log debug to stdout

* Wrong output

* remove stupid constants

* Fix lint and style issues
2021-04-29 22:42:18 -05:00
Tony Murray 1c379dcd05
Remove legacy function calls (#12651)
* massive inlines

* fix style and wtf

* remove rrdtool.inc.php include

* fix CommonFunctions namespace issues

* looking for missing class space, fix undefined class issues

* style fixes
2021-03-29 00:25:30 +02:00
Tony Murray 0d61deb8f0
Remove perf_times table (#12517)
the data is never used... This type of data is better to store in rrd anyway
2021-02-09 12:47:05 -06:00
Tony Murray 1e4702fa4f
Support multiple daily process locking backends with distributed polling (#11896)
* Implement locks in the file cache

* Replace custom locks

* implement restore lock
Used when re-hydrating

* remove legacy use statements

* Add class descriptions

* Fix style

* Default to database cache driver

* missed cache_locks table
prevent chicken-egg issue

* style fixes

* Remove custom file lock implementation

* missed items from file cache

* Update schema definition
hmm, other schema noise must be from manual modification as this is generated from a freshly migrated db.

* require predis, it is pure python, so no harm in adding

* and set predis as the default client
2020-10-07 07:36:35 -05:00
Jellyfrog 29f45ca352
Apply fixes from StyleCI (#12124) 2020-09-21 15:59:34 +02:00
Jellyfrog 0d56bbd946
Apply fixes from StyleCI (#12123) 2020-09-21 15:51:53 +02:00
Jellyfrog 77c531527c
Apply fixes from StyleCI (#12117)
* Apply fixes from StyleCI

* Disable style check
2020-09-21 14:54:51 +02:00
Tony Murray d5a52ca4eb
Fix various issues with loading os definitions (#11640)
* Ping only device doesn't display
if os was set to something, ping os wasn't loaded and we try to get overview graphs from it.

* Fix snmp_disable device page load error
When other os is set.

* Revamp os setting loading
the only safe way to access is Config::getOsSetting()

* Remove getOsSetting fallback behavior
Most instances don't use it and it can have unexpected results Config::getOsSetting('blah', 'group') == 'librenms'

* refactor and remove unneeded load_os/loadOs calls now since getOsSetting automatically loads it.

* restore unix overview graphs, they are different
small cleanups

* fix
2020-05-19 14:35:32 -05:00
Tony Murray dec9a498ee Updated device cache support (#10795)
* Device Cache
also some additional device related models

* attribs from cache

* replace common attribute functions

* remove legacy cache usage
tidy up some collection manipulation
remove some unused or single use functions

* cleanup some items

* always return a device, to prevent bugs

* clear device cache when testing after each test

* fix double assignment

* Clean up function to take advantage of null object
2019-11-14 21:56:06 +00:00
Tony Murray 699aa8a042
Consolidate configuration settings and implement dynamic webui settings (#9809)
* initial db/defaults dump

* numeric values

* Remove $config['time']

* Use config_definitions.json

* try setting definition as a complete array

* a little more

* fix format

* WIP converting dynamic setting pages

* rewriting the webui

* remove legacy and add translations

* finish email section

* improve navigation
update js a bit

* Import the remaining existing settings

* Update backend still some wip
migration

* fix config loading (db not overriding defaults)

* some cleanup

* more array collapsing

* update settings

* Settings search

* add purge settings to ui, order groups and sections

* collapse more arrays

* Auth definitions WIP

* remove needless slash escapes

* add warning to json

* moving settings from defaults.inc

* remove slow_statistics

* Move rrdtool settings to the poller section

* Use translations more consistently

* removing more settings from defaults.inc.php

* show setting name in tooltip

* allow help text to be shown on mobile

* disable settings that are set in config.php

* Implement undo and reset to default.

* Vue.js functional

* Vue.js WIP

* Implement tabs component

* accordion WIP

* lodash ver update

* two items

* Accordion somewhat working

* hash navigation

* Refine Accordion

* Fix up tab styling a bit

* Cleaner tab selected property
Hide html while loading

* spinner?

* Icon support
property for accordion to active

* WIP

* Tabs/Sections from ajax

* Setting Component skeletons

* Dynamic Component resolution

* Basic functionality

* toggle class

* Refactor components

* translate tabs and accordions

* simple array attempt

* improve readonly tooltip

* array styling

* array value editing

* organize snmp info

* Handle initial tab/section in url

* Use Laravel to parse tab/section, dump old

* Draggable array entries

* v-tooltip, for clickable (and touch) help tooltips
disable draggable

* Navigation WIP

* Navigation WIP

* groups computed

* filter settings

* fix event reference

* vue.js i18n initial

* missing description = just setting name

* en fallback

* tidy up the language support and js generation

* persist value to db

* fix issue with 0

* Delete settings from DB instead of setting them to default

* ldap-groups
fixup style

* Default dashboard selection

* fix array of options instead of object

* allow custom validation for settings

* translate options in SettingSelect

* SNMP v3 WIP

* fix setting arrays

* Split persist out of set

* Hook up events for SNMP v3 Auth
correct Config::persist behaviour with arrays

* dependent settings (primitive for now)
actually update the settings values in the LibrenmsSettings component

* more complex "when" behaviour

* remove un-needed seeder

* add poller_modules.cisco-qfp

* remove en.json (disable warning)

* don't set default for log_dir or log_file, otherwise it won't be processed correctly

* Fix module order
add some missing settings

* more config corrections

* correct graphs
correct loading values set to null (although it should have no difference)
remove project_name_version

* Add nfsen settings.  Docs are very confusing, so might have flubbed something
remove option for array definition of select option as numeric indexes messes it up

* Correct more upstream config differences

* Config cleanup after a bunch of merges.

* Fixes

* add version tags to js/css files
remove old js

* Print out full settings list read-only

* Add http_proxy setting
fix indents in config_definitions.json

* repeaters default is 0 (aka 20)

* cleanups

* rewrite the dynamic config docs

* add language docs

* Don't show snmp v3 auth add/remove if disabled by config.php
2019-10-16 21:22:05 +00:00
Tony Murray f3ba8947f7
Use Config helper (#10339)
remove usage of global variable
2019-06-23 00:29:12 -05:00
Tony Murray eeb3d58f5b Improved Logging and Debugging (#8870)
Use Log facility when Laravel is booted.
Update init.php so we can easily boot Laravel for CLI scripts. (and just Eloquent, but that may go away)
Move all debug setup into set_debug() function and use that across all scripts.
Log Laravel database queries.
Send debug output to librenms log file when enabling debug in the webui.
Allow for colorized Log CLI output. (currently will leave % tags in log file output)

** Needs testing and perhaps tweaking still.

DO NOT DELETE THIS TEXT

#### Please note

> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.

- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)

#### Testers

If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
2018-07-13 23:08:00 +01:00
Tony Murray 846cf11331 Support specifying submodules in poller and discovery for debug (#8896)
Mostly useful for debugging

DO NOT DELETE THIS TEXT

#### Please note

> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.

- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)

#### Testers

If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
2018-07-12 21:30:39 +01:00
costasd 33887417ee discovery.php: don't exit(5) without new devices discovered (#8893)
Don't exit with a non-zero value in case no new devices were discovered, as non-zero codes are considered failed jobs for most cron implementations and produce logs like the following (in Debian):

```
cron.info CRON[963890]: (CRON) error (grandchild #963892 failed with exit status 5)
``` 

DO NOT DELETE THIS TEXT

#### Please note

> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.

- [X] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)

#### Testers

If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
2018-07-10 21:41:24 +01:00
Tony Murray 0ba76e6d62 New python service for poller, discovery + more (#8455)
Currently has a file handle leak (and will eventually run out of handles) related to the self update process.

Either need to fix that or rip out self-update and leave that up to cron or something.


DO NOT DELETE THIS TEXT

#### Please note

> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.

- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)

#### Testers

If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
2018-06-30 12:19:49 +01:00
Neil Lathwood ecbbc0e3e5 feature: Clear OS defs cache on poller/discovery debug (#8061) 2018-01-09 10:51:26 -06:00
Tony Murray 2e73b75297 fix: Use memcached to lock daily processes on Distributed Pollers (#7735)
* fix: use memcached to lock daily processes on Distributed Pollers

* All the locks!
2017-11-24 09:37:52 +00:00
Tony Murray 9113f81272 fix: Allow discovery to check if devices are down before skipping (#7780)
* fix: allow discovery to check if devices are down before skipping
We could also exclude snmp_disable devices from SQL query, worthwhile?

* also skip ping only devices
2017-11-23 08:31:15 +00:00
Tony Murray 54b476c769
fix: devices detected as ibmtl or generic (#7618)
* fix: devices detected as ibmtl
Guess at the sysObjectID, please report if you have devices that should be ibmtl and are no longer detected.
Add output of os in discovery.

* Always load all os for discovery.  Should be cached most of the time.
2017-11-03 16:10:24 -05:00
Tony Murray 51ba934e11 feature: Added web validation (#7474)
* refactor: convert validations to modules to prep for gui integration

* accidentally dropped, schema update

* fix accidental output to webui

* mention discovery-wrapper.py and re-arrange.

* refine some fix statements

* rename the Config validation group to Configuration.

* fix some scrutinizer issues
remove as many local functions from validator.php as possible
move extensions from pre-check
remove duplicate timezone check
looks like there is some db schema differences between mariadb 10.1 and 10.2, investigating

* mariadb version diff take2

* Check schema version first for database.
Remove stop to go back to command line for install docs.
Add helpful link when there is no devices added to /addhost

* fix incorrect validation for empty string defaults

* Fix style

* Add additional file permissions checks

* catch exception and fail for invalid timezone
Change visuals a bit.

* add php version warning

* fix space
2017-10-26 07:56:09 +01:00
Tony Murray 181b0fb7d5 refactor: Use the new locks for schema updates (#6931)
* Use the new locks for schema updates

* removed schema lock from discovery

* Add the ability to wait for a lock.
Add unit tests

* Add MysqlLock
Use that for the schema updates
Wait up to 30s for other schema updates to complete.

* Switch the schema lock back to a file lock for now.
Make FileLock support indefinite locking without polling.
Add a warning to MysqlLock for scenarios where it won't work.

* Delete MysqlLock.php

* Removed MySQL lock tests
2017-07-17 19:35:08 +01:00
keeperofdakeys 701fbbc29b feature: Add a new locking framework that uses flock. (#6858)
Change locks to use flock, as pid checking is not
sufficient when PID Namespaces are involved.
2017-07-02 23:44:36 -05:00
Tony Murray b6e18db766 fix: page/graph load speed: part 1 (#6611)
* fix: page/graph load speed: part 1
Speed up load_all_os() by caching the full $config['os'] array
Function time goes from 1.13648s -> 0.00073s
The cache is updated during discovery, but only if it is >22hrs old.
Additionally, the cache is not used and a full yaml load is forced if the cache is >24hrs old.
The cache time is controlled by $config['os_def_cache_time'], I don't see any reason anyone would change that time.

* Delete Session.php

* Include the cache directory.
2017-05-10 08:57:10 +01:00
Neil Lathwood 5f6279d28c refactor: Added lock support to ./discovery.php -h new to prevent overlap (#6568) 2017-05-03 20:23:45 -05:00
Neil Lathwood 41e7689c77 refactor: DB Updates will now file level lock to stop duplicate updates (#6469)
* refactor: DB Updates will now file level lock to stop duplicate updates

* scrut fixes

* renamed function from clear_lock() to release_lock()
2017-05-03 14:48:23 +01:00
Tony Murray 374a4041af fix: inconsistent device discovery (#6518)
There were two or more separate $device arrays during discovery. When one was updated, others were not.
Combine those all.
Make sure the new os is loaded if the os changed.
2017-04-28 22:50:00 +01:00
Neil Lathwood 153f0de12f fix: Actually reload oxidized when we should not when we think we should (#6515) 2017-04-27 22:31:48 -05:00
Neil Lathwood d49e893a0a move version info into debug to save exec calls under normal use (#6407) 2017-04-22 08:10:15 -05:00
Neil Lathwood d5296319fb refactor: MySQL strict and query fixes (#5338)
* refactor: MySQL strict and query fixes

* moved sql file
2017-01-13 06:47:16 -06:00
Tony Murray db240cb4eb refactor: Collection and output of db and snmp stats (#5049) 2016-11-23 06:57:19 +00:00
Tony Murray 9a33464c52 refactor: Centralize includes and initialization (#4991) 2016-11-21 20:12:59 +00:00
Tony Murray d2193f76ca feature: allow scripts to be run from any working directory (#4437)
* feature: allow validate.php to be run from any working directory

* remove redundant realpath() call

* re-add realpath() to remove symbolic links

* realpath() isn't needed...

* chdir() in all php scripts in ./ and ./scripts/

* update-sql.php clearly is not used, as it was broken.

* Change some scripts to executable
Remove extra chdir() call in snmp-scan.php

* Missed console-ui.php
Kind of fixed console-ui.php help output

* Re-add newline
2016-09-14 16:53:04 +01:00
Neil Lathwood 7dec670494 refactor: Updated a lot of references to LibreNMS - copyright still left intact (#4339) 2016-09-08 08:12:23 -05:00
Tony Murray abc6a5b799 PSR-2 Final cleanup (#4247)
refactor: Final PSR2 cleanup
2016-08-28 23:32:55 +01:00
pblasquez be9db5a19e Update discovery.php 2016-02-24 12:48:22 -08:00
laf 0558275414 Added verbose debugging, normal debugging will now try and mask certain information 2016-01-17 14:39:17 +00:00
Neil Lathwood 72cc180184 Merge pull request #2719 from SaaldjorMike/del-utime-func
Replaced custom utime() function with microtime(true)
2016-01-10 22:43:31 +00:00
Mike Rostermund e2d1a8e7d9 Replaced custom utime() function with microtime(true) 2016-01-08 13:33:32 +01:00
laf fdc0eca44d Centralised version info + used in output 2016-01-06 00:14:35 +00:00
Daniel Preussker c7a55f86c6 Merge pull request #2093 from SaaldjorMike/remove-old-versioncheck
Removed old unused version checking
2015-10-09 18:04:13 +00:00