[MWCI-221] Setup appropriate certificate expiration warning probes for internal services Created: 27/Feb/20  Updated: 06/Jul/23

Status: Reopened
Project: Continuous Integration Infrastructure for Middleware Development
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Blocker
Reporter: Andrea Ceccanti Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Today at 12 the ci.cloud and repo.cloud certficates have expired, after 3 years.

Unfortunately we didn't have any probe warning us of this.

This issue will be used to track corrective actions to ensure that we have a proper warning system in place to watch for certificate expiration, until we move to let's encrypt for as many services as possible

 Comments   
Comment by Andrea Ceccanti [ 13/Sep/21 ]

Today we found out that chnet had an expired certificate (and that we do not have a probe for the problem). But chnet was in the list of services to be monitored (see comments above).

How comes no warning was in place for chnet?

Comment by Marcelo Vilaça Pinheiro Soares [X] (Inactive) [ 06/Oct/20 ]

Checked Uchiwa dashboard

Comment by Marcelo Vilaça Pinheiro Soares [X] (Inactive) [ 05/Oct/20 ]

Followed procedure described in:
https://issues.infn.it/jira/browse/CNSD-64?focusedCommentId=88149&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-88149

 

Comment by Marcelo Vilaça Pinheiro Soares [X] (Inactive) [ 05/Oct/20 ]

Added iam-eosc, iam-mvm, iam-super, iam-demo, iam-chnet

to /mw-devel/puppet-site/profiles/sensu_server.pp

and the respective URLs

 

 

Comment by Andrea Ceccanti [ 02/Oct/20 ]

I don't see the subscriptions, i.e. the probes are not run

Comment by Marcelo Vilaça Pinheiro Soares [X] (Inactive) [ 02/Oct/20 ]

Added to /mw-devel/sensu-mw-devel/mwdevel_sensu/manifests/setup/checks.pp

# EOSC
mwdevel_sensu::setup::check { 'check-eosc-iam-ssl':
command => 'check-ssl-host.rb --host iam-eosc.cloud.cnaf.infn.it --warning 30 --critical 14',
subscribers => 'iam-eosc',
}

# MVM
mwdevel_sensu::setup::check { 'check-mvm-iam-ssl':
command => 'check-ssl-host.rb --host iam-mvm.cloud.cnaf.infn.it --warning 30 --critical 14',
subscribers => 'iam-mvm',
}

# SUPER
mwdevel_sensu::setup::check { 'check-super-iam-ssl':
command => 'check-ssl-host.rb --host iam-super.cloud.cnaf.infn.it --warning 30 --critical 14',
subscribers => 'iam-super',
}

# DEMO
mwdevel_sensu::setup::check { 'check-demo-iam-ssl':
command => 'check-ssl-host.rb --host iam-demo.cloud.cnaf.infn.it --warning 30 --critical 14',
subscribers => 'iam-demo',
}

## CHNET
mwdevel_sensu::setup::check { 'check-iam-chnet-ssl':
command => 'check-ssl-host.rb --host chnet-iam.cloud.cnaf.infn.it --warning 30 --critical 14',
subscribers => 'iam-chnet',
}

Comment by Andrea Ceccanti [ 28/Sep/20 ]

10:30 $ sh utils/show-iam-deployed-versions.sh

chnet/chnet-iam-759cc56549-8rzfc: indigoiam/iam-login-service:v1.4.0-latest,
cnaf/iam-667fc9bcb9-cqchb: indigoiam/iam-login-service:v1.6.0-SNAPSHOT-c01b38d,
deep/iam-58d98cbbcd-ggl2v: indigoiam/iam-login-service:v1.6.0-SNAPSHOT-c01b38d,
demo/demo-iam-login-service-665f5d66c8-fptvq: indigoiam/iam-login-service:v1.5.0.RELEASE-latest,
dodas/dodas-iam-74d446f646-kplfj: indigoiam/iam-login-service:v1.6.0-SNAPSHOT-19336a0,
eosc/eosc-iam-login-service-66d8d59dbc-rdhm9: indigoiam/iam-login-service:v1.6.0-SNAPSHOT-c01b38d,
escape/escape-iam-login-service-54d498d7b-8542p: indigoiam/iam-login-service:v1.6.0,
indigo/iam-7c75c6d98-8vjzw: indigoiam/iam-login-service:v1.6.0-SNAPSHOT-c01b38d,
iotwins/iotwins-iam-login-service-79f58d7b95-p8k8m: indigoiam/iam-login-service:v1.6.0-SNAPSHOT-c01b38d,
mvm/mvm-iam-login-service-6cf78ffcf6-jchbm: indigoiam/iam-login-service:v1.6.0-SNAPSHOT-c01b38d,
pillar/pillar-iam-login-service-575cfdcdc8-flv6l: indigoiam/iam-login-service:v1.6.0-SNAPSHOT-c01b38d,
super/super-iam-login-service-864c76b977-2gs5c: indigoiam/iam-login-service:v1.6.0-SNAPSHOT-c01b38d,
t1-computing/t1-computing-iam-login-service-6b698d567c-qjwtt: indigoiam/iam-login-service:v1.6.0-SNAPSHOT-c01b38d,
virgo/virgo-iam-login-service-74d74d5dcc-5zr85: indigoiam/iam-login-service:v1.6.0-SNAPSHOT-c01b38d,
wlcg/wlcg-iam-login-service-74df99d679-z2lvb: indigoiam/iam-login-service:v1.6.0

Comment by Andrea Ceccanti [ 28/Sep/20 ]

Verify that a probe is in place for all IAM instances:

e.g. https://chnet-iam.cloud.cnaf.infn.it/login

Comment by Andrea Ceccanti [ 07/Sep/20 ]

Ensure we have probe in place for all services on the K8S cluster

Generated at Sun Aug 24 09:21:24 CEST 2025 using Jira 10.3.6#10030006-sha1:0dc21a711362757421d62af2e50bcb9585207f88.