couchdb-infra-cm.git
2 months agoReplaced the PPC64 VM with a faster one main
Nick Vatamaniuc [Thu, 30 Jun 2022 21:33:36 +0000 (17:33 -0400)] 
Replaced the PPC64 VM with a faster one

2 months agoUse Java 11 for Jenkins
Nick Vatamaniuc [Thu, 30 Jun 2022 19:23:18 +0000 (15:23 -0400)] 
Use Java 11 for Jenkins

2 months agoReplace PPC64LE worker with a new VM
Nick Vatamaniuc [Mon, 27 Jun 2022 17:43:59 +0000 (13:43 -0400)] 
Replace PPC64LE worker with a new VM

3 months agoPowerVS instance was replaced
Nick Vatamaniuc [Thu, 23 Jun 2022 22:17:01 +0000 (18:17 -0400)] 
PowerVS instance was replaced

7 months agoAdd a few more examples how to upgrade and restart jenkins workers
Nick Vatamaniuc [Fri, 18 Feb 2022 15:25:20 +0000 (10:25 -0500)] 
Add a few more examples how to upgrade and restart jenkins workers

10 months agoImprove formatting in ssh.cfg
Nick Vatamaniuc [Mon, 15 Nov 2021 19:06:46 +0000 (14:06 -0500)] 
Improve formatting in ssh.cfg

Add a space after each section

10 months agocouchdb01 host_vars files didn't have a .yml extension
Nick Vatamaniuc [Mon, 15 Nov 2021 19:03:20 +0000 (14:03 -0500)] 
couchdb01 host_vars files didn't have a .yml extension

10 months agoImprove bastion setup
Nick Vatamaniuc [Fri, 12 Nov 2021 23:17:09 +0000 (18:17 -0500)] 
Improve bastion setup

 * Use a single control socket.
   - Use the bastion hostname in proxy commands instead of IP
   - CI agents which need bastions don't start an extra control socket

 * Avoid using a relative ./ssh.cfg file
   - Instead users can include it in their `~/.ssh/config` file
   - Now it is possible to ssh in without having to be in the source directory

13 months agoReplace power instance coucdhb-ci4 -> couchdb-ci-ubuntu-2
Nick Vatamaniuc [Thu, 19 Aug 2021 18:39:35 +0000 (14:39 -0400)] 
Replace power instance coucdhb-ci4 -> couchdb-ci-ubuntu-2

The other one failed probably after a bad package upgrade and reboot. It got
stuck in cloud network init code.

To prevent this new instance from doing the same, added a config file to
/etc/cloud/cloud.cfg.d to disable auto cloud network setup:

```
-- 95_disable_network_config.cfg --
network:
  config: disabled
```

14 months agoUpdate host_vars for the new PowerVS instance
Nick Vatamaniuc [Wed, 7 Jul 2021 22:23:11 +0000 (18:23 -0400)] 
Update host_vars for the new PowerVS instance

14 months agoUpdate PowerVS instance
Nick Vatamaniuc [Wed, 7 Jul 2021 22:08:33 +0000 (18:08 -0400)] 
Update PowerVS instance

The previous instance got locked out and froze during boot.

It seems that /etc/cloud/cloud.cfg is to blame. We have to ensure that
```
    network:
      config: disabled
```

is set before every reboot.

14 months agoUpdate playbooks to set up power and s390x hosts
Nick Vatamaniuc [Mon, 5 Jul 2021 22:35:21 +0000 (18:35 -0400)] 
Update playbooks to set up power and s390x hosts

Hosts are Ubuntu 20.04 so to avoid forking all the scripts added `become: yes`
to most commands. In a few cases had to check for Ubuntu vs Debian, but it
wasn't too bad overall. Perhaps eventually we could just use Ubuntu everywhere
to simplify things since it seems to be the lowest common denominator.

s390x host `couchdb01` seems to come up:
```
Open Blue Ocean
Build Executor Status
1 Idle
2 Idle
Inbound agent connected from 148.100.113.138/148.100.113.138:52506
Remoting version: 4.5
This is a Unix agent
Agent successfully connected and online
```

PowerVS host `couchdb-ci-ubuntu-2` has not come back up after playbook install.
So may need aditional debugging.

14 months agoAdd host vars for PowerVS and s390x instance
Nick Vatamaniuc [Fri, 2 Jul 2021 23:49:09 +0000 (19:49 -0400)] 
Add host vars for PowerVS and s390x instance

And delete the now defunct ppc64le

14 months agoAdd the ability to add random unmanaged instances
Nick Vatamaniuc [Fri, 2 Jul 2021 21:22:04 +0000 (17:22 -0400)] 
Add the ability to add random unmanaged instances

The s390x host as such an example

14 months agoReplace PowerVS instance
Nick Vatamaniuc [Fri, 2 Jul 2021 18:58:47 +0000 (14:58 -0400)] 
Replace PowerVS instance

It was broken after a bad apt-get upgrade

14 months agoUpdate PowerVS instance from CentOS 8 to Ubuntu 20.04
Nick Vatamaniuc [Mon, 28 Jun 2021 21:06:34 +0000 (17:06 -0400)] 
Update PowerVS instance from CentOS 8 to Ubuntu 20.04

Ubuntu 20.04 is closer to Debian which we already use and CentOS will stop
1-to-1 comptable with RHEL soon enough.

15 months agoGenerate Ansible SSH config for Power instance
Nick Vatamaniuc [Mon, 14 Jun 2021 23:04:13 +0000 (19:04 -0400)] 
Generate Ansible SSH config for Power instance

15 months agoFinalize configuring power instances
Nick Vatamaniuc [Mon, 14 Jun 2021 22:37:07 +0000 (18:37 -0400)] 
Finalize configuring power instances

Configuration API is bit different than the regular VPC instances, so we they
ended up a separate hard-coded config environment named "power".

15 months agoUpdate .asf.yaml to avoid spamming dev@couchdb mailing list
Nick Vatamaniuc [Mon, 14 Jun 2021 14:44:20 +0000 (10:44 -0400)] 
Update .asf.yaml to avoid spamming dev@couchdb mailing list

15 months agoMerge pull request #1 from apache/multiple-environments
Nick Vatamaniuc [Fri, 11 Jun 2021 20:28:34 +0000 (16:28 -0400)] 
Merge pull request #1 from apache/multiple-environments

Make it possible to discover instances from various IBM cloud accounts

15 months agoUpdate worker debian-dal-1-08's IP address 1/head
Nick Vatamaniuc [Fri, 11 Jun 2021 20:25:12 +0000 (16:25 -0400)] 
Update worker debian-dal-1-08's IP address

This one was recreated not too long ago and we forgot to commit the changes.

15 months agoRead configurations from multiple IBM cloud environments
Nick Vatamaniuc [Thu, 13 May 2021 05:01:31 +0000 (01:01 -0400)] 
Read configurations from multiple IBM cloud environments

Config file can look like:
```
[ibmcloud.env1]
api_key = ...

[ibmcloud.env2]
api_key = ...
```

Then instances are gathered from all environments.

For now only x86 instances are gathered, in the future we'll gather powerVS
instances as well.

Also convert to python3

21 months agoEnsure root account is locked
Paul J. Davis [Mon, 14 Dec 2020 21:11:50 +0000 (15:11 -0600)] 
Ensure root account is locked

Its locked by default but we might as well add the extra protection just
in case.

21 months agoHarden SSH settings on CI nodes
Paul J. Davis [Mon, 14 Dec 2020 20:41:59 +0000 (14:41 -0600)] 
Harden SSH settings on CI nodes

21 months agoUpdate inventory after rebuilding the farm
Paul J. Davis [Fri, 4 Dec 2020 18:12:59 +0000 (12:12 -0600)] 
Update inventory after rebuilding the farm

IBM needed me to recreate the CI farm because they had block storage
instances that needed updating to a never version.

2 years agoUpdate ansible inventory
Paul J. Davis [Thu, 18 Jun 2020 19:12:06 +0000 (14:12 -0500)] 
Update ansible inventory

2 years agoGrant access to Nick Vatamaniuc
Paul J. Davis [Thu, 18 Jun 2020 18:15:32 +0000 (13:15 -0500)] 
Grant access to Nick Vatamaniuc

2 years agoAdd ad hoc command example
Paul J. Davis [Thu, 18 Jun 2020 18:11:32 +0000 (13:11 -0500)] 
Add ad hoc command example

2 years agoAdd two ppc64le hosts
Paul J. Davis [Mon, 13 Jan 2020 15:11:39 +0000 (09:11 -0600)] 
Add two ppc64le hosts

These Power machines require Ubuntu for Docker support.

2 years agoRework Docker for ppc64le support
Paul J. Davis [Mon, 13 Jan 2020 15:11:19 +0000 (09:11 -0600)] 
Rework Docker for ppc64le support

2 years agoMake Ansible got faster
Paul J. Davis [Thu, 9 Jan 2020 23:47:46 +0000 (17:47 -0600)] 
Make Ansible got faster

Still not what I would call fast though.

2 years agoOnly run docker tasks on x86_64 hosts
Paul J. Davis [Thu, 9 Jan 2020 23:16:30 +0000 (17:16 -0600)] 
Only run docker tasks on x86_64 hosts

2 years agoConfigure TCP Keep-Alive parameters
Paul J. Davis [Thu, 9 Jan 2020 19:32:36 +0000 (13:32 -0600)] 
Configure TCP Keep-Alive parameters

2 years agoGive kocolosk access to CI nodes
Paul J. Davis [Wed, 8 Jan 2020 19:06:11 +0000 (13:06 -0600)] 
Give kocolosk access to CI nodes

2 years agoUpdate README.md
Paul J. Davis [Wed, 8 Jan 2020 19:05:33 +0000 (13:05 -0600)] 
Update README.md

2 years agoSwitch to having agents dial into Jenkins
Paul J. Davis [Tue, 7 Jan 2020 21:41:55 +0000 (15:41 -0600)] 
Switch to having agents dial into Jenkins

This changes the configuration of Jenkins agents to dial into the
Jenkins master instead of having Jenkins SSH to each node. This allows
us to expand our private cloud worker pool much further.

2 years agoUpdated scripts and configuration
Paul J. Davis [Wed, 11 Dec 2019 17:47:22 +0000 (11:47 -0600)] 
Updated scripts and configuration

3 years agoAdd new roles and provision playbooks
Paul J. Davis [Wed, 14 Aug 2019 19:09:34 +0000 (14:09 -0500)] 
Add new roles and provision playbooks

3 years agoBasic inventory generation
Paul J. Davis [Fri, 9 Aug 2019 19:50:19 +0000 (14:50 -0500)] 
Basic inventory generation

For the time being I'm relying on a tool to generate our inventory
lists. In the future if we get our permissions scheme setup we'll be
able to open this up to anyone that has an IBM Cloud account that is
associated with the CouchDB CI Infra account.

3 years agoInitial commit
Paul J. Davis [Fri, 9 Aug 2019 18:28:55 +0000 (13:28 -0500)] 
Initial commit