Datagerry: Cannot connect with Mongod. Tested working connection with mongosh

sink1 · April 30, 2024, 9:40am

Hi,

The cmdb.conf has this:

[Database]
host = db1.x.y
port = 27017
database_name = datagerry
username = datagerry_user
password = 1234
#tls = true

I can connect successfully from the same server with:

# mongosh "mongodb://db1.x.y/datagerry" --username=datagerry_user --password  123 --authenticationDatabase datagerry
Current Mongosh Log ID: 6630b7951f8e8c6565b5a386
Connecting to:          mongodb://db1.x.y/datagerry?directConnection=true
Using MongoDB:          7.0.7
Using Mongosh:          1.1.6
For mongosh info see: https://docs.mongodb.com/mongodb-shell/
datagerry> show tables;
new_collection
datagerry>

Starting mongodb says:

Apr 30 11:08:36 cmdb1 datagerry[73542]:        Welcome to DATAGERRY
Apr 30 11:08:36 cmdb1 datagerry[73542]:        Starting system with following parameters:
Apr 30 11:08:36 cmdb1 datagerry[73542]:        {'keys': False, 'debug': False, 'start': True, 'config_file': '/etc/datagerry/cmdb.conf'}
Apr 30 11:08:36 cmdb1 datagerry[73542]:
Apr 30 11:08:36 cmdb1 datagerry[73542]:        Copyright (C) 2024 becon GmbH
Apr 30 11:08:36 cmdb1 datagerry[73542]:        licensed under the terms of the GNU Affero General Public License version 3
Apr 30 11:08:36 cmdb1 datagerry[73542]:
Apr 30 11:08:37 cmdb1 datagerry[73548]: [2024-04-30 11:08:37][INFO    ] --- DATAGERRY starting... (__main__.py)
Apr 30 11:08:37 cmdb1 datagerry[73548]: [2024-04-30 11:08:37][INFO    ] --- Checking database connection with cmdb.conf data (__main__.py)
Apr 30 11:08:37 cmdb1 datagerry[73548]: [2024-04-30 11:08:37][CRITICAL] --- CMDBError: Authentication failed., full error: {'ok': 0.0, 'errmsg': 'Authentication failed.', 'code': 18, 'codeName': 'AuthenticationFailed'} (__main__.py)
Apr 30 11:08:37 cmdb1 datagerry[73548]: [2024-04-30 11:08:37][INFO    ] --- DATAGERRY stopped! (__main__.py)

sink1 · April 30, 2024, 12:06pm

Error mesgae in mongodb logs:


{"t":{"$date":"2024-04-30T13:51:38.708+02:00"},"s":"I",  "c":"ACCESS",   "id":20436,   "ctx":"conn115","msg":"Checking authorization failed","attr":{"error":{"code":13,"codeName":"Unauthorized","errmsg":"not authorized on datagerry to execute command { usersInfo: { user: \"current_username\", db: \"datagerry\" }, lsid: { id: UUID(\"10208145-91bb-4f7c-9eae-d78fb754c092\") }, $db: \"datagerry\" }"}}}
{"t":{"$date":"2024-04-30T13:54:00.634+02:00"},"s":"I",  "c":"ACCESS",   "id":20251,   "ctx":"conn117","msg":"Supported SASL mechanisms requested for unknown user","attr":{"user":{"user":"datagerry_user","db":"admin"}}}
{"t":{"$date":"2024-04-30T13:54:00.634+02:00"},"s":"I",  "c":"ACCESS",   "id":5286307, "ctx":"conn117","msg":"Failed to authenticate","attr":{"client":"1.2.3.4:51988","isSpeculative":true,"isClusterMember":false,"mechanism":"SCRAM-SHA-256","user":"datagerry_user","db":"admin","error":"UserNotFound: Could not find user \"datagerry_user\" for db \"admin\"","result":11,"metrics":{"conversation_duration":{"micros":355,"summary":{"0":{"step":1,"step_total":2,"duration_micros":339}}}},"extraInfo":{}}}

sink1 · April 30, 2024, 12:34pm

What permissions does the mongdodb user require?

I have got this:

datagerry> db.getUser('datagerry_user')
{
  _id: 'datagerry.datagerry_user',
  userId: UUID("b2e99cb9-0bf3-4576-8dc3-2fc88a8f7d7b"),
  user: 'datagerry_user',
  db: 'datagerry',
  roles: [ { role: 'readWrite', db: 'datagerry' } ],
  mechanisms: [ 'SCRAM-SHA-1', 'SCRAM-SHA-256' ]
}

sink1 · April 30, 2024, 12:34pm

Are my posts read by anybody?

MiB · May 2, 2024, 8:29am

Hi,
this is my (default) entry from cmdb.conf:

[Database]
host = 127.0.0.1
port = 27017
database_name = cmdb
;username = username
;password = password

So, use the anonymous user feature, that is: leave the username/pssword settings commented.

sink1 · May 2, 2024, 8:36am

Hi,

How does this work when mongod.conf has security enabled? We cannot give anonymous users access to a dB server. Compliance and legal issues aside, the Security department would have kittens

security:
  authorization: enabled

datagerry> db.getUsers()
{
  users: [
    {
      _id: 'datagerry.datagerry_user',
      userId: UUID("xx-xx-xx-xx-xx"),
      user: 'datagerry_user',
      db: 'datagerry',
      roles: [ { role: 'readWrite', db: 'datagerry' } ],
      mechanisms: [ 'SCRAM-SHA-1', 'SCRAM-SHA-256' ]
    }
  ],
  ok: 1
}

sink1 · May 2, 2024, 8:57am

cmdb1 datagerry[88291]: [2024-05-02 10:55:01][INFO    ] --- DATAGERRY starting... (__main__.py)
 cmdb1 datagerry[88291]: [2024-05-02 10:55:01][INFO    ] --- Checking database connection with cmdb.conf data (__main__.py)
cmdb1 datagerry[88291]: [2024-05-02 10:55:01][CRITICAL] --- CMDBError: Authentication failed., full error: {'ok': 0.0, 'errmsg': 'Authentication failed.', 'code': 18, 'codeName': 'AuthenticationFailed'} (__main__.py)
cmdb1 datagerry[88291]: [2024-05-02 10:55:01][INFO    ] --- DATAGERRY stopped! (__main__.py)

Does Datagerry work with SCRAM-SHA-1 or SCRAM-SHA-256 password mechanisms?

The reason I think this is because I can connect from the datagerry sever to the database server:

mongosh "mongodb://pfbsdbsrvnp218.alphacredit.acgroup/datagerry" --username=datagerry_user --password  xxxxxxxxxx --authenticationDatabase datagerry
Current Mongosh Log ID: 663356f62c9bec4728a03e92
Connecting to:          mongodb://pfbsdbsrvnp218.alphacredit.acgroup/datagerry?directConnection=true
Using MongoDB:          7.0.7
Using Mongosh:          1.1.6
For mongosh info see: https://docs.mongodb.com/mongodb-shell/
------
   The server generated these startup warnings when booting:
   2024-nn-nnTnn:12:53.934+02:00: vm.max_map_count is too low
------
datagerry>

Thus we know that the connection details and firewall flows are correct, but datagerry is unable to connect with these settings from the same server.

sink1 · May 2, 2024, 9:09am

For a test I disabled the username and password and it worked but it did threw some python errors. How significant are these?

cmdb1 datagerry[88758]: [INFO] --- UPDATE ROUTINE: Update database collection finished. (__update__.py)
cmdb1 datagerry[88758]: [INFO] --- UPDATE ROUTINE: FINISHED! (__update__.py)
cmdb1 datagerry[88758]: [INFO] --- SETUP ROUTINE: STARTED... (__setup__.py)
cmdb1 datagerry[88758]: [INFO] --- SETUP ROUTINE: Checking database connection (__setup__.py)
cmdb1 datagerry[88758]: [INFO] --- SETUP ROUTINE: Database connection status True (__setup__.py)
cmdb1 datagerry[88758]: [INFO] --- SETUP ROUTINE: FINISHED! (__setup__.py)
cmdb1 datagerry[88781]: [INFO] --- start exportd ... (service.py)
cmdb1 datagerry[88781]: [INFO] --- exportd: start run (service.py)
cmdb1 datagerry[88758]: [INFO] --- Process manager started: True (__main__.py)
cmdb1 datagerry[88758]: [INFO] --- DATAGERRY successfully started (__main__.py)
cmdb1 datagerry[88786]: [INFO] --- start webapp ... (service.py)
cmdb1 datagerry[88786]: [INFO] --- Interfaces started @ http://0.0.0.0:4000 (gunicorn.py)
cmdb1 datagerry[88781]: Exception in thread Thread-2:
cmdb1 datagerry[88781]: Traceback (most recent call last):
cmdb1 datagerry[88781]:  File "threading.py", line 980, in _bootstrap_inner
cmdb1 datagerry[88781]: Exception in thread Thread-1:
cmdb1 datagerry[88781]: Traceback (most recent call last):
cmdb1 datagerry[88781]:  File "cmdb/event_management/event_manager.py", line 321, in run
cmdb1 datagerry[88781]:  File "cmdb/event_management/event_manager.py", line 293, in __init_connection
cmdb1 datagerry[88781]:  File "pika/adapters/blocking_connection.py", line 377, in __init__
cmdb1 datagerry[88781]:  File "threading.py", line 980, in _bootstrap_inner
cmdb1 datagerry[88781]:  File "pika/adapters/blocking_connection.py", line 416, in _process_io_for_connection_setup
cmdb1 datagerry[88781]:  File "pika/adapters/blocking_connection.py", line 469, in _flush_output
cmdb1 datagerry[88781]:  File "cmdb/event_management/event_manager.py", line 199, in run
cmdb1 datagerry[88781]:  File "pika/adapters/base_connection.py", line 174, in _adapter_connect
cmdb1 datagerry[88781]:  File "pika/adapters/base_connection.py", line 304, in _getaddrinfo
cmdb1 datagerry[88781]:  File "socket.py", line 954, in getaddrinfo
cmdb1 datagerry[88781]:  File "cmdb/event_management/event_manager.py", line 178, in __init_connection
cmdb1 datagerry[88781]: socket.gaierror: [Errno -2] Name or service not known
cmdb1 datagerry[88781]:  File "pika/adapters/blocking_connection.py", line 377, in __init__
cmdb1 datagerry[88781]:  File "pika/adapters/blocking_connection.py", line 416, in _process_io_for_connection_setup
cmdb1 datagerry[88781]:  File "pika/adapters/blocking_connection.py", line 469, in _flush_output
cmdb1 datagerry[88781]:  File "pika/adapters/base_connection.py", line 174, in _adapter_connect
cmdb1 datagerry[88781]:  File "pika/adapters/base_connection.py", line 304, in _getaddrinfo
cmdb1 datagerry[88781]:  File "socket.py", line 954, in getaddrinfo
cmdb1 datagerry[88781]: socket.gaierror: [Errno -2] Name or service not known
cmdb1 datagerry[88786]: Exception in thread Thread-2:
cmdb1 datagerry[88786]: Traceback (most recent call last):
cmdb1 datagerry[88786]:  File "threading.py", line 980, in _bootstrap_inner
cmdb1 datagerry[88786]:  File "cmdb/event_management/event_manager.py", line 199, in run
cmdb1 datagerry[88786]:  File "cmdb/event_management/event_manager.py", line 178, in __init_connection
cmdb1 datagerry[88786]:  File "pika/adapters/blocking_connection.py", line 377, in __init__
cmdb1 datagerry[88786]:  File "pika/adapters/blocking_connection.py", line 416, in _process_io_for_connection_setup
cmdb1 datagerry[88786]:  File "pika/adapters/blocking_connection.py", line 469, in _flush_output
cmdb1 datagerry[88786]:  File "pika/adapters/base_connection.py", line 174, in _adapter_connect
cmdb1 datagerry[88786]:  File "pika/adapters/base_connection.py", line 304, in _getaddrinfo
cmdb1 datagerry[88786]:  File "socket.py", line 954, in getaddrinfo
cmdb1 datagerry[88786]: socket.gaierror: [Errno -2] Name or service not known
cmdb1 datagerry[88786]: Exception in thread Thread-3:
cmdb1 datagerry[88786]: Traceback (most recent call last):
cmdb1 datagerry[88786]:  File "threading.py", line 980, in _bootstrap_inner
cmdb1 datagerry[88786]:  File "cmdb/event_management/event_manager.py", line 321, in run
cmdb1 datagerry[88786]:  File "cmdb/event_management/event_manager.py", line 293, in __init_connection
cmdb1 datagerry[88786]:  File "pika/adapters/blocking_connection.py", line 377, in __init__
cmdb1 datagerry[88786]:  File "pika/adapters/blocking_connection.py", line 416, in _process_io_for_connection_setup
cmdb1 datagerry[88786]:  File "pika/adapters/blocking_connection.py", line 469, in _flush_output
cmdb1 datagerry[88786]:  File "pika/adapters/base_connection.py", line 174, in _adapter_connect
cmdb1 datagerry[88786]:  File "pika/adapters/base_connection.py", line 304, in _getaddrinfo
cmdb1 datagerry[88786]:  File "socket.py", line 954, in getaddrinfo
cmdb1 datagerry[88786]: socket.gaierror: [Errno -2] Name or service not known

However, it is listening:

# lsof -i tcp:4000 -P
COMMAND     PID      USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
datagerry 88791 datagerry   40u  IPv4 975664      0t0  TCP *:4000 (LISTEN)
datagerry 88792 datagerry   40u  IPv4 975664      0t0  TCP *:4000 (LISTEN)
datagerry 88793 datagerry   40u  IPv4 975664      0t0  TCP *:4000 (LISTEN)
datagerry 88794 datagerry   40u  IPv4 975664      0t0  TCP *:4000 (LISTEN)
datagerry 88795 datagerry   40u  IPv4 975664      0t0  TCP *:4000 (LISTEN)
datagerry 88796 datagerry   40u  IPv4 975664      0t0  TCP *:4000 (LISTEN)

sink1 · May 2, 2024, 9:36am

Datagerry web gui is accessible. We have a reverse proxy in front of it.

The DataGerry documentation website states:
https://datagerry.readthedocs.io/en/latest/admin_guide/setup.html#setup-datagerry

To access the DATAGERRY frontend, use the following parameters:
http://<>:4000
user: admin
password: admin

But these credentials did not work. The admin credentials were refused. I have a 640kB video showing the error, but this website won’t let us post video. Photograph below.

Here are the login logs, but I did not notice an error related to this per se:
( I have put the debug nginx logs taken during the attempted login further down this post. )

# tail -f /var/log/nginx/cmdb1.x.y.local.*
==> /var/log/nginx/cmdb1.x.y.local.error.log <==
==> /var/log/nginx/cmdb1.x.y.local.access.log <==
10.223.33.126 - - [02/May/2024:11:42:40 +0200] "POST /rest/auth/login HTTP/1.1" 401 424 "https://cmdb1.x.y/auth" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36 Edg/122.0.0.0"

These are the mongodb messages that appear when I try to login with admin|admin:

  "id":22943,   "ctx":"listener","msg":"Connection accepted","attr":{"remote":"10.80.23.201:52148","uuid":{"uuid":{"$uuid":"2d4e94fb-d633-43ee-ba39-988a9155ee1c"}},"connectionId":164,"connectionCount":34}}
  "id":51800,   "ctx":"conn164","msg":"client metadata","attr":{"remote":"10.80.23.201:52148","client":"conn164","negotiatedCompressors":[],"doc":{"driver":{"name":"PyMongo","version":"4.6.0"},"os":{"type":"Linux","name":"Linux","architecture":"x86_64","version":"5.14.0-362.18.1.el9_3.x86_64"},"platform":"CPython 3.9.16.final.0"}}}
  "id":22943,   "ctx":"listener","msg":"Connection accepted","attr":{"remote":"10.80.23.201:52154","uuid":{"uuid":{"$uuid":"bfdc9bd7-1cb0-4db8-86af-da413923d867"}},"connectionId":165,"connectionCount":35}}
  "id":22943,   "ctx":"listener","msg":"Connection accepted","attr":{"remote":"10.80.23.201:52162","uuid":{"uuid":{"$uuid":"de3aead3-ddfa-44d1-8053-923523b66237"}},"connectionId":166,"connectionCount":36}}
  "id":51800,   "ctx":"conn165","msg":"client metadata","attr":{"remote":"10.80.23.201:52154","client":"conn165","negotiatedCompressors":[],"doc":{"driver":{"name":"PyMongo","version":"4.6.0"},"os":{"type":"Linux","name":"Linux","architecture":"x86_64","version":"5.14.0-362.18.1.el9_3.x86_64"},"platform":"CPython 3.9.16.final.0"}}}
  "id":51800,   "ctx":"conn166","msg":"client metadata","attr":{"remote":"10.80.23.201:52162","client":"conn166","negotiatedCompressors":[],"doc":{"driver":{"name":"PyMongo","version":"4.6.0"},"os":{"type":"Linux","name":"Linux","architecture":"x86_64","version":"5.14.0-362.18.1.el9_3.x86_64"},"platform":"CPython 3.9.16.final.0"}}}
  "id":6788700, "ctx":"conn165","msg":"Received first command on ingress connection since session start or auth handshake","attr":{"elapsedMillis":0}

And the nginx access and error logs show:

 [debug] 89427#89427: *18 http keepalive handler
 [debug] 89427#89427: *18 malloc: 00005559A40B8930:1024
 [debug] 89427#89427: *18 SSL_read: 753
 [debug] 89427#89427: *18 SSL_read: -1
 [debug] 89427#89427: *18 SSL_get_error: 2
 [debug] 89427#89427: *18 reusable connection: 0
 [debug] 89427#89427: *18 posix_memalign: 00005559A40CA680:4096 @16
 [debug] 89427#89427: *18 event timer del: 17: 858996854
 [debug] 89427#89427: *18 http process request line
 [debug] 89427#89427: *18 http request line: "POST /rest/auth/login HTTP/1.1"
 [debug] 89427#89427: *18 http uri: "/rest/auth/login"
 [debug] 89427#89427: *18 http args: ""
 [debug] 89427#89427: *18 http exten: ""
 [debug] 89427#89427: *18 posix_memalign: 00005559A40A8C70:4096 @16
 [debug] 89427#89427: *18 http process request header line
 [debug] 89427#89427: *18 http header: "Host: cmdb1.x.y"
 [debug] 89427#89427: *18 http header: "Connection: keep-alive"
 [debug] 89427#89427: *18 http header: "Content-Length: 40"
 [debug] 89427#89427: *18 http header: "sec-ch-ua: "Chromium";v="122", "Not(A:Brand";v="24", "Microsoft Edge";v="122""
 [debug] 89427#89427: *18 http header: "Accept: application/json, text/plain, */*"
 [debug] 89427#89427: *18 http header: "Content-Type: application/json"
 [debug] 89427#89427: *18 http header: "sec-ch-ua-mobile: ?0"
 [debug] 89427#89427: *18 http header: "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36 Edg/122.0.0.0"
 [debug] 89427#89427: *18 http header: "sec-ch-ua-platform: "Windows""
 [debug] 89427#89427: *18 http header: "Origin: https://cmdb1.x.y"
 [debug] 89427#89427: *18 http header: "Sec-Fetch-Site: same-origin"
 [debug] 89427#89427: *18 http header: "Sec-Fetch-Mode: cors"
 [debug] 89427#89427: *18 http header: "Sec-Fetch-Dest: empty"
 [debug] 89427#89427: *18 http header: "Referer: https://cmdb1.x.y/auth"
 [debug] 89427#89427: *18 http header: "Accept-Encoding: gzip, deflate, br"
 [debug] 89427#89427: *18 http header: "Accept-Language: en-US,en;q=0.9"
 [debug] 89427#89427: *18 http header done
 [debug] 89427#89427: *18 generic phase: 0
 [debug] 89427#89427: *18 rewrite phase: 1
 [debug] 89427#89427: *18 test location: "/"
 [debug] 89427#89427: *18 using configuration "/"
 [debug] 89427#89427: *18 http cl:40 max:1048576
 [debug] 89427#89427: *18 rewrite phase: 3
 [debug] 89427#89427: *18 http script value: "http://127.0.0.1:4000"
 [debug] 89427#89427: *18 http script set $upstream_host
 [debug] 89427#89427: *18 http script regex: "^/(.*)"
 [notice] 89427#89427: *18 "^/(.*)" matches "/rest/auth/login", client: 10.223.33.126, server: _, request: "POST /rest/auth/login HTTP/1.1", host: "cmdb1.x.y", referrer: "https://cmdb1.x.y/auth"
 [debug] 89427#89427: *18 http script copy: "/"
 [debug] 89427#89427: *18 http script capture: "rest/auth/login"
 [debug] 89427#89427: *18 http script regex end
 [notice] 89427#89427: *18 rewritten data: "/rest/auth/login", args: "", client: 10.223.33.126, server: _, request: "POST /rest/auth/login HTTP/1.1", host: "cmdb1.x.y", referrer: "https://cmdb1.x.y/auth"
 [debug] 89427#89427: *18 post rewrite phase: 4
 [debug] 89427#89427: *18 generic phase: 5
 [debug] 89427#89427: *18 generic phase: 6
 [debug] 89427#89427: *18 generic phase: 7
 [debug] 89427#89427: *18 generic phase: 8
 [debug] 89427#89427: *18 access phase: 9
 [debug] 89427#89427: *18 access phase: 10
 [debug] 89427#89427: *18 access phase: 11
 [debug] 89427#89427: *18 post access phase: 12
 [debug] 89427#89427: *18 generic phase: 13
 [debug] 89427#89427: *18 generic phase: 14
 [debug] 89427#89427: *18 http script var: "http://127.0.0.1:4000"
 [debug] 89427#89427: *18 http client request body preread 40
 [debug] 89427#89427: *18 http request body content length filter
 [debug] 89427#89427: *18 http body new buf t:1 f:0 00005559A40B8BF9, pos 00005559A40B8BF9, size: 40 file: 0, size: 0
 [debug] 89427#89427: *18 http init upstream, client timer: 0
 [debug] 89427#89427: *18 http script copy: "Host"
 [debug] 89427#89427: *18 http script var: "cmdb1.x.y"
 [debug] 89427#89427: *18 http script copy: "X-Real-IP"
 [debug] 89427#89427: *18 http script var: "10.223.33.126"
 [debug] 89427#89427: *18 http script copy: "X-Forwarded-For"
 [debug] 89427#89427: *18 http script var: "10.223.33.126"
 [debug] 89427#89427: *18 http script copy: "X-Forwarded-Proto"
 [debug] 89427#89427: *18 http script var: "https"
 [debug] 89427#89427: *18 http script copy: "Connection"
 [debug] 89427#89427: *18 http script copy: "close"
 [debug] 89427#89427: *18 http script copy: "Content-Length"
 [debug] 89427#89427: *18 http script var: "40"
 [debug] 89427#89427: *18 http script copy: ""
 [debug] 89427#89427: *18 http proxy header: "sec-ch-ua: "Chromium";v="122", "Not(A:Brand";v="24", "Microsoft Edge";v="122""
 [debug] 89427#89427: *18 http proxy header: "Accept: application/json, text/plain, */*"
 [debug] 89427#89427: *18 http proxy header: "Content-Type: application/json"
 [debug] 89427#89427: *18 http proxy header: "sec-ch-ua-mobile: ?0"
 [debug] 89427#89427: *18 http proxy header: "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36 Edg/122.0.0.0"
 [debug] 89427#89427: *18 http proxy header: "sec-ch-ua-platform: "Windows""
 [debug] 89427#89427: *18 http proxy header: "Origin: https://cmdb1.x.y"
 [debug] 89427#89427: *18 http proxy header: "Sec-Fetch-Site: same-origin"
 [debug] 89427#89427: *18 http proxy header: "Sec-Fetch-Mode: cors"
 [debug] 89427#89427: *18 http proxy header: "Sec-Fetch-Dest: empty"
 [debug] 89427#89427: *18 http proxy header: "Referer: https://cmdb1.x.y/auth"
 [debug] 89427#89427: *18 http proxy header: "Accept-Encoding: gzip, deflate, br"
 [debug] 89427#89427: *18 http proxy header: "Accept-Language: en-US,en;q=0.9"
 [debug] 89427#89427: *18 http proxy header:
"POST /rest/auth/login HTTP/1.0
Host: cmdb1.x.y
X-Real-IP: 10.223.33.126
X-Forwarded-For: 10.223.33.126
X-Forwarded-Proto: https
Connection: close
Content-Length: 40
sec-ch-ua: "Chromium";v="122", "Not(A:Brand";v="24", "Microsoft Edge";v="122"
Accept: application/json, text/plain, */*
Content-Type: application/json
sec-ch-ua-mobile: ?0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36 Edg/122.0.0.0
sec-ch-ua-platform: "Windows"
Origin: https://cmdb1.x.y
Sec-Fetch-Site: same-origin
Sec-Fetch-Mode: cors
Sec-Fetch-Dest: empty
Referer: https://cmdb1.x.y/auth
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9

"
 [debug] 89427#89427: *18 http cleanup add: 00005559A40A9C20
 [debug] 89427#89427: *18 posix_memalign: 00005559A41CF6D0:4096 @16
 [debug] 89427#89427: *18 get rr peer, try: 1
 [debug] 89427#89427: *18 stream socket 19
 [debug] 89427#89427: *18 epoll add connection: fd:19 ev:80002005
 [debug] 89427#89427: *18 connect to 127.0.0.1:4000, fd:19 #47
 [debug] 89427#89427: *18 http upstream connect: -2
 [debug] 89427#89427: *18 posix_memalign: 00005559A3FF29C0:128 @16
 [debug] 89427#89427: *18 event timer add: 19: 60000:859016064
 [debug] 89427#89427: *18 http finalize request: -4, "/rest/auth/login?" a:1, c:2
 [debug] 89427#89427: *18 http request count:2 blk:0
 [debug] 89427#89427: *18 http run request: "/rest/auth/login?"
 [debug] 89427#89427: *18 http upstream check client, write event:1, "/rest/auth/login"
 [debug] 89427#89427: *18 http upstream request: "/rest/auth/login?"
 [debug] 89427#89427: *18 http upstream send request handler
 [debug] 89427#89427: *18 http upstream send request
 [debug] 89427#89427: *18 http upstream send request body
 [debug] 89427#89427: *18 chain writer buf fl:0 s:792
 [debug] 89427#89427: *18 chain writer buf fl:1 s:40
 [debug] 89427#89427: *18 chain writer in: 00005559A41CF908
 [debug] 89427#89427: *18 writev: 832 of 832
 [debug] 89427#89427: *18 chain writer out: 0000000000000000
 [debug] 89427#89427: *18 event timer del: 19: 859016064
 [debug] 89427#89427: *18 event timer add: 19: 60000:859016065
 [debug] 89427#89427: *18 http upstream request: "/rest/auth/login?"
 [debug] 89427#89427: *18 http upstream process header
 [debug] 89427#89427: *18 malloc: 00005559A41BDF30:4096
 [debug] 89427#89427: *18 recv: eof:0, avail:-1
 [debug] 89427#89427: *18 recv: fd:19 302 of 4096
 [debug] 89427#89427: *18 http proxy status 401 "401 UNAUTHORIZED"
 [debug] 89427#89427: *18 http proxy header: "Server: gunicorn"
 [debug] 89427#89427: *18 http proxy header: "Date: Thu, 02 May 2024 11:06:52 GMT"
 [debug] 89427#89427: *18 http proxy header: "Connection: close"
 [debug] 89427#89427: *18 http proxy header: "Content-Type: application/json"
 [debug] 89427#89427: *18 http proxy header: "Content-Length: 424"
 [debug] 89427#89427: *18 http proxy header: "Access-Control-Allow-Origin: https://cmdb1.x.y"
 [debug] 89427#89427: *18 http proxy header: "Access-Control-Expose-Headers: X-API-Version, X-Total-Count"
 [debug] 89427#89427: *18 http proxy header: "Vary: Origin"
 [debug] 89427#89427: *18 http proxy header done
 [debug] 89427#89427: *18 xslt filter header
 [debug] 89427#89427: *18 HTTP/1.1 401 UNAUTHORIZED
Server: nginx/1.20.1
Date: Thu, 02 May 2024 11:06:52 GMT
Content-Type: application/json
Content-Length: 424
Connection: keep-alive
Access-Control-Allow-Origin: https://cmdb1.x.y
Access-Control-Expose-Headers: X-API-Version, X-Total-Count
Vary: Origin

 [debug] 89427#89427: *18 write new buf t:1 f:0 00005559A41CFCE8, pos 00005559A41CFCE8, size: 311 file: 0, size: 0
 [debug] 89427#89427: *18 http write filter: l:0 f:0 s:311
 [debug] 89427#89427: *18 http cacheable: 0
 [debug] 89427#89427: *18 http proxy filter init s:401 h:0 c:0 l:424
 [debug] 89427#89427: *18 http upstream process upstream
 [debug] 89427#89427: *18 pipe read upstream: 0
 [debug] 89427#89427: *18 pipe preread: 0
 [debug] 89427#89427: *18 pipe buf free s:0 t:1 f:0 00005559A41BDF30, pos 00005559A41BE05E, size: 0 file: 0, size: 0
 [debug] 89427#89427: *18 pipe length: 424
 [debug] 89427#89427: *18 event timer: 19, old: 859016065, new: 859016070
 [debug] 89427#89427: *18 http upstream request: "/rest/auth/login?"
 [debug] 89427#89427: *18 http upstream dummy handler
 [debug] 89427#89427: *18 http upstream request: "/rest/auth/login?"
 [debug] 89427#89427: *18 http upstream process upstream
 [debug] 89427#89427: *18 pipe read upstream: 1
 [debug] 89427#89427: *18 readv: eof:0, avail:-1
 [debug] 89427#89427: *18 readv: 1, last:3794
 [debug] 89427#89427: *18 pipe recv chain: 424
 [debug] 89427#89427: *18 pipe buf free s:0 t:1 f:0 00005559A41BDF30, pos 00005559A41BE05E, size: 424 file: 0, size: 0
 [debug] 89427#89427: *18 pipe length: 424
 [debug] 89427#89427: *18 input buf #0
 [debug] 89427#89427: *18 pipe write downstream: 1
 [debug] 89427#89427: *18 pipe write downstream flush in
 [debug] 89427#89427: *18 http output filter "/rest/auth/login?"
 [debug] 89427#89427: *18 http copy filter: "/rest/auth/login?"
 [debug] 89427#89427: *18 image filter
 [debug] 89427#89427: *18 xslt filter body
 [debug] 89427#89427: *18 http postpone filter "/rest/auth/login?" 00005559A41CF918
 [debug] 89427#89427: *18 write old buf t:1 f:0 00005559A41CFCE8, pos 00005559A41CFCE8, size: 311 file: 0, size: 0
 [debug] 89427#89427: *18 write new buf t:1 f:0 00005559A41BDF30, pos 00005559A41BE05E, size: 424 file: 0, size: 0
 [debug] 89427#89427: *18 http write filter: l:0 f:0 s:735
 [debug] 89427#89427: *18 http copy filter: 0 "/rest/auth/login?"
 [debug] 89427#89427: *18 pipe write downstream done
 [debug] 89427#89427: *18 event timer: 19, old: 859016065, new: 859016070
 [debug] 89427#89427: *18 http upstream exit: 0000000000000000
 [debug] 89427#89427: *18 finalize http upstream request: 0
 [debug] 89427#89427: *18 finalize http proxy request
 [debug] 89427#89427: *18 free rr peer 1 0
 [debug] 89427#89427: *18 close http upstream connection: 19
 [debug] 89427#89427: *18 free: 00005559A3FF29C0, unused: 48
 [debug] 89427#89427: *18 event timer del: 19: 859016065
 [debug] 89427#89427: *18 reusable connection: 0
 [debug] 89427#89427: *18 http upstream temp fd: -1
 [debug] 89427#89427: *18 http output filter "/rest/auth/login?"
 [debug] 89427#89427: *18 http copy filter: "/rest/auth/login?"
 [debug] 89427#89427: *18 image filter
 [debug] 89427#89427: *18 xslt filter body
 [debug] 89427#89427: *18 http postpone filter "/rest/auth/login?" 00007FFF3F668E30
 [debug] 89427#89427: *18 write old buf t:1 f:0 00005559A41CFCE8, pos 00005559A41CFCE8, size: 311 file: 0, size: 0
 [debug] 89427#89427: *18 write old buf t:1 f:0 00005559A41BDF30, pos 00005559A41BE05E, size: 424 file: 0, size: 0
 [debug] 89427#89427: *18 write new buf t:0 f:0 0000000000000000, pos 0000000000000000, size: 0 file: 0, size: 0
 [debug] 89427#89427: *18 http write filter: l:1 f:0 s:735
 [debug] 89427#89427: *18 http write filter limit 0
 [debug] 89427#89427: *18 malloc: 00005559A41C4150:16384
 [debug] 89427#89427: *18 SSL buf copy: 311
 [debug] 89427#89427: *18 SSL buf copy: 424
 [debug] 89427#89427: *18 SSL to write: 735
 [debug] 89427#89427: *18 SSL_write: 735
 [debug] 89427#89427: *18 http write filter 0000000000000000
 [debug] 89427#89427: *18 http copy filter: 0 "/rest/auth/login?"
 [debug] 89427#89427: *18 http finalize request: 0, "/rest/auth/login?" a:1, c:1
 [debug] 89427#89427: *18 set http keepalive handler
 [debug] 89427#89427: *18 http close request
 [debug] 89427#89427: *18 http log handler

==> /var/log/nginx/cmdb1.x.y.local.access.log <==
10.223.33.126 - - [02/May/2024:13:06:52 +0200] "POST /rest/auth/login HTTP/1.1" 401 424 "https://cmdb1.x.y/auth" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36 Edg/122.0.0.0"

==> /var/log/nginx/cmdb1.x.y.local.error.log <==
 [debug] 89427#89427: *18 free: 00005559A41BDF30
 [debug] 89427#89427: *18 free: 00005559A40CA680, unused: 3
 [debug] 89427#89427: *18 free: 00005559A40A8C70, unused: 0
 [debug] 89427#89427: *18 free: 00005559A41CF6D0, unused: 1032
 [debug] 89427#89427: *18 free: 00005559A40B8930
 [debug] 89427#89427: *18 hc free: 0000000000000000
 [debug] 89427#89427: *18 hc busy: 0000000000000000 0
 [debug] 89427#89427: *18 free: 00005559A41C4150
 [debug] 89427#89427: *18 reusable connection: 1
 [debug] 89427#89427: *18 event timer add: 17: 65000:859021070

Can anybody see anything amiss?

sink1 · May 2, 2024, 1:33pm

Hi,

Was datagerry tested with mongodb7.0.7 by becon GmbH?

I ask because the datagerry requirements have this:

DATAGERRY has the following system requirements:

Linux Operating System

MongoDB 4.4+ (MongoDB 6.0 recommended)

RabbitMQ (except the deb-packages)

sink1 · May 6, 2024, 8:56am

Hi,

I removed mongod 7.0.7 and installed version 6.0.15:

# rpm -qa|grep mongo
mongodb-org-mongos-6.0.15-1.el9.x86_64
mongodb-org-server-6.0.15-1.el9.x86_64
mongodb-mongosh-2.2.5-1.el8.x86_64

And enabled authentication and created the user.

MongoSH can connect from the datagerry server as show here:

# mongosh "mongodb://datagerry_user:redacted@db1:27017/datagerry" --authenticationDatabase "datagerry"
Current Mongosh Log ID: 66389ad977b0f57ab5a99637
Connecting to:          mongodb://<credentials>@db1:27017/datagerry?directConnection=true
Using MongoDB:          6.0.15
Using Mongosh:          1.1.6
For mongosh info see: https://docs.mongodb.com/mongodb-shell/
datagerry> show tables;
new_collection
datagerry>

/etc/datagerry/cmdb.conf has this for the database:

# cat /etc/datagerry/cmdb.conf
[Database]
host = db1
port = 27017
database_name = datagerry
username = datagerry_user
password = redacted
tls=false

Datagerry did not work:

datagerry[206831]: [2024-05-06 10:51:33][INFO    ] --- DATAGERRY starting... (__main__.py)
May  6 10:51:33 acrmsasrv225 datagerry[206831]: [2024-05-06 10:51:33][INFO    ] --- Checking database connection with cmdb.conf data (__main__.py)
datagerry[206831]: [2024-05-06 10:51:33][CRITICAL] --- CMDBError: Authentication failed., full error: {'ok': 0.0, 'errmsg': 'Authentication failed.', 'code': 18, 'codeName': 'AuthenticationFailed'} (__main__.py)
datagerry[206831]: [2024-05-06 10:51:33][INFO    ] --- DATAGERRY stopped! (__main__.py)

I started several threads last week about running datagerry on RHEL9, mongodb with authenitcation enabled, and nobody really replied.

I suspect that Datagerry will not work on mongodb with any authentication enabled, and I also wonder if MongodB works on RHEL9, but have not got any proof.

The deafening silence for the Datagerry community makes me very suspicious.

sink1 · May 6, 2024, 9:02am

I cannot justify buying paid support from Becon gmbh if Datagerry won’t work on RHEL9, or with MongodB with authentication enabled.

adnan.smajic · May 7, 2024, 6:59am

Hi @sink1,
we will have a datiled look at your issue and improve the documentation regarding your questions in upcoming releases.

Currently DATAGERRY is not tested with MongoDB 7 but will be probably added in the near future.

DATAGERRY works with RHEL9.

BR Adnan

sink1 · May 8, 2024, 1:09pm

Got it working with RHEL9, but not only with mongo v6 installed locally.

Cannot get it working across the network, which we have to fix to go into PRD.

Have not managed to get RabbitMQ working.


#  rabbitmqctl list_permissions --vhost datagerry
Listing permissions for vhost "datagerry" ...
user    configure       write   read
datagerry       .       .       .*

But get this error:

`NOT_ALLOWED - access to vhost ‘/’ refused for user ‘datagerry’

Here is the config:

adnan.smajic · May 14, 2024, 7:12am

We currently dont have your setup as an test environment and therefore it is hard to help you out there. Have your checked the official RabbitMQ forums for setup via VMs?

BR Adnan

sink1 · May 14, 2024, 7:14am

Hi,

I changed the vhost settings for datagerry on Rabbitmq from . . .* to .* .* .*

I did get a login, but for some reason datagerry is trying to app / instead of the datagerry vhost on our test rabbbitmq server. Below is the configuration. Of note is that it will work if I use the guest user with access to /, but this is not actually permitted in a production environment nor best practive.

Does my following cmdb.conf look correct to access a RabbitMq vhost called datagerry?

[MessageQueueing]
host = 10.1.1.3
#host = 127.0.0.1
port = 5672
username = datagerry
password = redacted
#username = guest
#password = redacted
exchange = datagerry.eventbus
connection_attempts = 2
retry_delay = 6
;use_tls = true

Log file is here showing messages when I start datagerry up?

 08:25:03.461 [info] <0.12057.14> Successfully set permissions for 'datagerry' in virtual host 'datagerry' to '.*', '.*', '.*'
 09:05:46.777 [info] <0.15121.14> accepting AMQP connection <0.15121.14> (cmdb1.x.y:41306 -> rabbit1.x.y:5672)
 09:05:46.777 [info] <0.15124.14> accepting AMQP connection <0.15124.14> (cmdb1.x.y:41312 -> rabbit1.x.y:5672)
 09:05:46.780 [error] <0.15121.14> Error on AMQP connection <0.15121.14> (cmdb1.x.y:41306 -> rabbit1.x.y:5672, user: 'datagerry', state: opening):
access to vhost '/' refused for user 'datagerry'
 09:05:46.781 [error] <0.15124.14> Error on AMQP connection <0.15124.14> (cmdb1.x.y:41312 -> rabbit1.x.y:5672, user: 'datagerry', state: opening):
access to vhost '/' refused for user 'datagerry'
 09:05:46.781 [info] <0.15121.14> closing AMQP connection <0.15121.14> (cmdb1.x.y:41306 -> rabbit1.x.y:5672, vhost: 'none', user: 'datagerry')
 09:05:46.782 [info] <0.15124.14> closing AMQP connection <0.15124.14> (cmdb1.x.y:41312 -> rabbit1.x.y:5672, vhost: 'none', user: 'datagerry')
 09:05:46.783 [info] <0.15132.14> Closing all channels from connection 'cmdb1.x.y:41306 -> rabbit1.x.y:5672' because it has been closed
 09:05:46.783 [info] <0.15134.14> Closing all channels from connection 'cmdb1.x.y:41312 -> rabbit1.x.y:5672' because it has been closed
 09:05:47.032 [info] <0.15152.14> accepting AMQP connection <0.15152.14> (cmdb1.x.y:41322 -> rabbit1.x.y:5672)
 09:05:47.038 [error] <0.15152.14> Error on AMQP connection <0.15152.14> (cmdb1.x.y:41322 -> rabbit1.x.y:5672, user: 'datagerry', state: opening):
access to vhost '/' refused for user 'datagerry'
 09:05:47.039 [info] <0.15152.14> closing AMQP connection <0.15152.14> (cmdb1.x.y:41322 -> rabbit1.x.y:5672, vhost: 'none', user: 'datagerry')
 09:05:47.040 [info] <0.15160.14> Closing all channels from connection 'cmdb1.x.y:41322 -> rabbit1.x.y:5672' because it has been closed
 09:05:47.042 [info] <0.15155.14> accepting AMQP connection <0.15155.14> (cmdb1.x.y:41336 -> rabbit1.x.y:5672)
 09:05:47.056 [error] <0.15155.14> Error on AMQP connection <0.15155.14> (cmdb1.x.y:41336 -> rabbit1.x.y:5672, user: 'datagerry', state: opening):
access to vhost '/' refused for user 'datagerry'
 09:05:47.066 [info] <0.15155.14> closing AMQP connection <0.15155.14> (cmdb1.x.y:41336 -> rabbit1.x.y:5672, vhost: 'none', user: 'datagerry')
 09:05:47.067 [info] <0.15165.14> Closing all channels from connection 'cmdb1.x.y:41336 -> rabbit1.x.y:5672' because it has been closed

adnan.smajic · May 14, 2024, 7:26am

The cmdb.conf is correct.

It looks like there is still some issue with the permissions for the user ‘datagerry’.

We will improve the Error logging for RabbitMQ in this or in the next release.

BR Adnan

Topic		Replies	Views
Advice sought on configuring Data Gerry with Mongodb over TLS with self-signed CA Community Support	17	167	November 27, 2024
Datagerry support configurations: Mongo 7.0.7 and RHEL9? Community Support	11	129	May 14, 2024
Datagerry and mongo relicaset Report a Bug	3	308	June 20, 2023
Locked out from datagerry Community Support	3	813	December 14, 2020
Datagerry 2.2.0 upgrade caused database connection errro Community Support	8	70	November 7, 2024

Datagerry: Cannot connect with Mongod. Tested working connection with mongosh

Related topics