Emc2Net - IT technology https://e-mc2.net/rss.xml en Elasticsearch in garbage collection hell https://e-mc2.net/elasticsearch-garbage-collection-hell <span property="schema:name">Elasticsearch in garbage collection hell</span> <span rel="schema:author"><span lang="" about="/user/1" typeof="schema:Person" property="schema:name" datatype="">rafaelma</span></span> <span property="schema:dateCreated" content="2019-05-30T14:56:53+00:00">Thu, 30/05/2019 - 16:56</span> <div class="field field--name-body field--type-text-with-summary field--label-above"> <div class="field__label">Body</div> <div property="schema:text" class="field__item"><p>After several weeks of intense testing, fixing configuration problems, re-indexing data and experiencing problems when upgrading our Kibana indices, we managed to upgrade our 36 Kibana instances and our Elasticsearch cluster in production from version 5.6.16 to 6.7.1 a couple of weeks ago.</p> <p>We could not believe it but finally we had over 1.300 indices, 100TB of data, 102 000 000 000 documents and 18 Elasticsearch nodes running the last version of the elastic 6.x series at the moment.</p></div> </div> Thu, 30 May 2019 14:56:53 +0000 rafaelma 119 at https://e-mc2.net https://e-mc2.net/elasticsearch-garbage-collection-hell#comments Keepalived - A documentation nightmare https://e-mc2.net/keepalived-documentation-nightmare <span property="schema:name">Keepalived - A documentation nightmare</span> <span rel="schema:author"><span lang="" about="/user/1" typeof="schema:Person" property="schema:name" datatype="">rafaelma</span></span> <span property="schema:dateCreated" content="2018-03-06T20:40:47+00:00">Tue, 06/03/2018 - 21:40</span> <div class="field field--name-body field--type-text-with-summary field--label-above"> <div class="field__label">Body</div> <div property="schema:text" class="field__item"><p>I am writing this article with contradictory feelings, Am I having a bad day on top of The Oracle at Google not giving me answers, or is the documentation of the Keepalived software totally outdated and old?</p></div> </div> Tue, 06 Mar 2018 20:40:47 +0000 rafaelma 118 at https://e-mc2.net https://e-mc2.net/keepalived-documentation-nightmare#comments Zabbix in a DevOps environment https://e-mc2.net/zabbix-devops-environment <span property="schema:name">Zabbix in a DevOps environment</span> <span rel="schema:author"><span lang="" about="/user/1" typeof="schema:Person" property="schema:name" datatype="">rafaelma</span></span> <span property="schema:dateCreated" content="2017-09-16T05:31:55+00:00">Sat, 16/09/2017 - 07:31</span> <div class="field field--name-body field--type-text-with-summary field--label-above"> <div class="field__label">Body</div> <div property="schema:text" class="field__item"><p>A case study about how the University of Oslo uses Zabbix in a DEVOPS environment where changes happen fast and no one can have control of everything.</p></div> </div> Sat, 16 Sep 2017 05:31:55 +0000 rafaelma 117 at https://e-mc2.net Elasticsearch - Common maintenance tasks https://e-mc2.net/elasticsearch-common-maintenance-tasks <span property="schema:name">Elasticsearch - Common maintenance tasks </span> <span rel="schema:author"><span lang="" about="/user/1" typeof="schema:Person" property="schema:name" datatype="">rafaelma</span></span> <span property="schema:dateCreated" content="2017-07-10T11:23:53+00:00">Mon, 10/07/2017 - 13:23</span> <div class="field field--name-body field--type-text-with-summary field--label-above"> <div class="field__label">Body</div> <div property="schema:text" class="field__item"><p>If you have to administrate an Elasticsearch cluster, there are some common maintenance tasks that you will have to run to keep your data growth under control, backup your indexes and keep the cluster updated.</p> <p>At the University of Oslo we have a 14 nodes Elasticsearch 5.x cluster (3 master + 2 clients + 4 SSD data+ 5 SAS data). We use it to manage, search, analyze, and explore our logs. It has around 100TB of total storage, around 1,300 indexes and we keep from 3 to 6-12 months of data per index type depending of the type of data they have. </p></div> </div> Mon, 10 Jul 2017 11:23:53 +0000 rafaelma 111 at https://e-mc2.net https://e-mc2.net/elasticsearch-common-maintenance-tasks#comments Access to Elasticsearch with Cerebro via SSL+LDAP https://e-mc2.net/access-elasticsearch-cerebro-sslldap <span property="schema:name">Access to Elasticsearch with Cerebro via SSL+LDAP</span> <span rel="schema:author"><span lang="" about="/user/1" typeof="schema:Person" property="schema:name" datatype="">rafaelma</span></span> <span property="schema:dateCreated" content="2017-06-25T13:53:42+00:00">Sun, 25/06/2017 - 15:53</span> <div class="field field--name-body field--type-text-with-summary field--label-above"> <div class="field__label">Body</div> <div property="schema:text" class="field__item"><p>One of the main plugins we were using with our 2.x Elasticsearch cluster was KOPF. This plugin was a web interface to the Elasticsearch API and it was an easy way of performing common tasks on our Elasticsearch cluster.</p> <p>When we upgraded our Elasticsearch cluster to version 5.x, we could not continue using this plugin because it was not longer supported. The good thing was that the author of KOPF, Leonardo Menezes, had a new project called "CEREBRO" to offer an alternative to KOPF when running Elasticsearch 5.x.</p></div> </div> Sun, 25 Jun 2017 13:53:42 +0000 rafaelma 99 at https://e-mc2.net https://e-mc2.net/access-elasticsearch-cerebro-sslldap#comments Using Zabbix with PostgreSQL as the database backend https://e-mc2.net/using-zabbix-postgresql-database-backend <span property="schema:name">Using Zabbix with PostgreSQL as the database backend</span> <span rel="schema:author"><span lang="" about="/user/1" typeof="schema:Person" property="schema:name" datatype="">rafaelma</span></span> <span property="schema:dateCreated" content="2017-06-19T19:22:34+00:00">Mon, 19/06/2017 - 21:22</span> <div class="field field--name-body field--type-text-with-summary field--label-above"> <div class="field__label">Body</div> <div property="schema:text" class="field__item"><p>We started using Zabbix to monitor the IT infrastructure at The University of Oslo in 2014. During all this time we have been running all our Zabbix servers on VMware virtual servers with an acceptable level of performance. This situation changed some months ago when the VMvare+Storage we were using did not have more available resources for us to grow and it was slowing down the future development of our monitoring system.</p></div> </div> Mon, 19 Jun 2017 19:22:34 +0000 rafaelma 97 at https://e-mc2.net PgBackMan - Release version 1.2.0 https://e-mc2.net/pgbackman-release-version-120 <span property="schema:name">PgBackMan - Release version 1.2.0</span> <span rel="schema:author"><span lang="" about="/user/1" typeof="schema:Person" property="schema:name" datatype="">rafaelma</span></span> <span property="schema:dateCreated" content="2017-06-13T21:36:07+00:00">Tue, 13/06/2017 - 23:36</span> <div class="field field--name-body field--type-text-with-summary field--label-above"> <div class="field__label">Body</div> <div property="schema:text" class="field__item"><p>It has been a while since the last release of <a href="/pgbackman">PgBackMan</a> and with this version we hope to start a new period with more frequent releases. This new release implements some new features and fixes some bugs from version 1.1.0. Source files, RPM and DEB files are available at <a href="https://github.com/rafaelma/pgbackman/releases">GitHub</a>.</p> <h2>New features</h2> <ul><li> <p>Add support for postgreSQL 9.6</p> </li> <li> <p>Add dbname exception field to register_snapshot_definition</p></div> </div> Tue, 13 Jun 2017 21:36:07 +0000 rafaelma 100 at https://e-mc2.net Zabbix-CLI - Release version 1.7.0 https://e-mc2.net/zabbix-cli-release-version-170 <span property="schema:name">Zabbix-CLI - Release version 1.7.0</span> <span rel="schema:author"><span lang="" about="/user/1" typeof="schema:Person" property="schema:name" datatype="">rafaelma</span></span> <span property="schema:dateCreated" content="2017-06-09T12:06:19+00:00">Fri, 09/06/2017 - 14:06</span> <div class="field field--name-body field--type-text-with-summary field--label-above"> <div class="field__label">Body</div> <div property="schema:text" class="field__item"><p>A new version of <a href="https://e-mc2.net/zabbix-cli">Zabbix-CLI</a> has been released to the public. Source files, RPM and DEB files are available at <a href="https://github.com/usit-gd/zabbix-cli/releases">GitHub</a>.</p> <p>This new version has new general improvements and new functionality to acknowledge events.</p> <h2>New features</h2> <ul><li> <p>[Issue:<a href="https://github.com/usit-gd/zabbix-cli/issues/17">#17</a>]: Add a new command 'show_trigger_events' to show events<br /> for a trigger.</p></div> </div> Fri, 09 Jun 2017 12:06:19 +0000 rafaelma 98 at https://e-mc2.net ZABBIX automation @ UiO https://e-mc2.net/zabbix-automation-uio <span property="schema:name">ZABBIX automation @ UiO</span> <span rel="schema:author"><span lang="" about="/user/1" typeof="schema:Person" property="schema:name" datatype="">rafaelma</span></span> <span property="schema:dateCreated" content="2017-05-02T12:59:14+00:00">Tue, 02/05/2017 - 14:59</span> <div class="field field--name-body field--type-text-with-summary field--label-above"> <div class="field__label">Body</div> <div property="schema:text" class="field__item"><p>A short presentation focused on how we automate the Zabbix configuration at the Universitity of Oslo</p></div> </div> Tue, 02 May 2017 12:59:14 +0000 rafaelma 114 at https://e-mc2.net https://e-mc2.net/zabbix-automation-uio#comments Examples from the new version of Zabbix-CLI - v.1.6.0 https://e-mc2.net/examples-new-version-zabbix-cli-v160 <span property="schema:name">Examples from the new version of Zabbix-CLI - v.1.6.0</span> <span rel="schema:author"><span lang="" about="/user/1" typeof="schema:Person" property="schema:name" datatype="">rafaelma</span></span> <span property="schema:dateCreated" content="2017-01-13T22:40:09+00:00">Fri, 13/01/2017 - 23:40</span> <div class="field field--name-body field--type-text-with-summary field--label-above"> <div class="field__label">Body</div> <div property="schema:text" class="field__item"><p>Just before Christmas we launched a new version of the Zabbix-CLI client we have developed at The University of Oslo.</p> <p>Zabbix-CLI 1.6.0 implements some new options for configuring the client and some new commands to handle maintenance periods in Zabbix via Zabbix-CLI.</p> <p>To read about all the possibilities available to configure this client, you can use the Zabbix-CLI manual found at Github: <a href="https://github.com/usit-gd/zabbix-cli/blob/master/docs/manual.rst#configuration">https://github.com/usit-gd/zabbix-cli/blob/master/docs/manual.rst#configuration</a></p></div> </div> Fri, 13 Jan 2017 22:40:09 +0000 rafaelma 88 at https://e-mc2.net Zabbix-CLI - Release version 1.6.0 https://e-mc2.net/zabbix-cli-release-version-160 <span property="schema:name">Zabbix-CLI - Release version 1.6.0</span> <span rel="schema:author"><span lang="" about="/user/1" typeof="schema:Person" property="schema:name" datatype="">rafaelma</span></span> <span property="schema:dateCreated" content="2016-12-06T17:03:47+00:00">Tue, 06/12/2016 - 18:03</span> <div class="field field--name-body field--type-text-with-summary field--label-above"> <div class="field__label">Body</div> <div property="schema:text" class="field__item"><p>A new version of <a href="https://e-mc2.net/zabbix-cli">Zabbix-CLI</a> has been released. Source files, RPM and DEB files are available at <a href="https://github.com/usit-gd/zabbix-cli/releases/tag/1.6.0">GitHub</a>.</p> <p>We are working to get Zabbix-CLI in the official Debian repository and a repository for RPM packages will be available soon.</p></div> </div> Tue, 06 Dec 2016 17:03:47 +0000 rafaelma 40 at https://e-mc2.net Lightning Talk - Zabbix-CLI https://e-mc2.net/lightning-talk-zabbix-cli <span property="schema:name">Lightning Talk - Zabbix-CLI</span> <span rel="schema:author"><span lang="" about="/user/1" typeof="schema:Person" property="schema:name" datatype="">rafaelma</span></span> <span property="schema:dateCreated" content="2016-10-09T10:00:00+00:00">Sun, 09/10/2016 - 12:00</span> <div class="field field--name-body field--type-text-with-summary field--label-above"> <div class="field__label">Body</div> <div property="schema:text" class="field__item"><p>This presentaion is a 5 minuttes lightning talk that presents Zabbix-CLI.</p> <p>Zabbix-cli is a tool for managing some administration tasks in monitoring systems running Zabbix.</p> <p>It is a terminal client written in Python that uses the Zabbix-API to connect to your Zabbix installation. It has been developed and tested by members of the Department for IT Infrastructure at the Center for Information Technology at the University of Oslo, Norway.</p></div> </div> Sun, 09 Oct 2016 10:00:00 +0000 rafaelma 52 at https://e-mc2.net Zabbix@UiO https://e-mc2.net/zabbixuio <span property="schema:name">Zabbix@UiO</span> <span rel="schema:author"><span lang="" about="/user/1" typeof="schema:Person" property="schema:name" datatype="">rafaelma</span></span> <span property="schema:dateCreated" content="2016-09-10T10:00:00+00:00">Sat, 10/09/2016 - 12:00</span> <div class="field field--name-body field--type-text-with-summary field--label-above"> <div class="field__label">Body</div> <div property="schema:text" class="field__item"><p>A case study showing the problems we have resolved with Zabbix and the challenges we had when we implemented Zabbix as the main monitoring tool at the University of Oslo.</p> <p>The number of challenges is not low in an organization as heterogenous as ours, with many thousands of servers and clients, all kinds of devices connected to our infrastructure, different operating systems, multiple locations and hundreds of IT staff. Full automation and delegation of privileges are the key words in the work we have done during the past year and a half.</p></div> </div> Sat, 10 Sep 2016 10:00:00 +0000 rafaelma 51 at https://e-mc2.net Sentralisert logging ved UiO https://e-mc2.net/sentralisert-logging-ved-uio <span property="schema:name">Sentralisert logging ved UiO</span> <span rel="schema:author"><span lang="" about="/user/1" typeof="schema:Person" property="schema:name" datatype="">rafaelma</span></span> <span property="schema:dateCreated" content="2016-04-21T10:00:00+00:00">Thu, 21/04/2016 - 12:00</span> <div class="field field--name-body field--type-text-with-summary field--label-above"> <div class="field__label">Body</div> <div property="schema:text" class="field__item"><p>A presentation about why and how we are using data analysis of log information at the University of Oslo with elasticsearch, logstash and kibana.</p></div> </div> Thu, 21 Apr 2016 10:00:00 +0000 rafaelma 54 at https://e-mc2.net PostgreSQL Backup Manager https://e-mc2.net/postgresql-backup-manager <span property="schema:name">PostgreSQL Backup Manager</span> <span rel="schema:author"><span lang="" about="/user/1" typeof="schema:Person" property="schema:name" datatype="">rafaelma</span></span> <span property="schema:dateCreated" content="2015-10-29T11:00:00+00:00">Thu, 29/10/2015 - 12:00</span> <div class="field field--name-body field--type-text-with-summary field--label-above"> <div class="field__label">Body</div> <div property="schema:text" class="field__item"><p>A 5 minuttes lightning talk that presents PgBackMan.</p> <p>PgBackMan is a tool for managing PostgreSQL logical backups created with pg_dump and pg_dumpall. It is designed to manage backups from thousands of databases running in multiple PostgreSQL nodes, and it supports a multiple backup server topology.</p> <p>It also manages role and database configuration information when creating a backup of a database.</p></div> </div> Thu, 29 Oct 2015 11:00:00 +0000 rafaelma 53 at https://e-mc2.net https://e-mc2.net/postgresql-backup-manager#comments