How to install TokenManager

[hanx-hep]

Hello,

I have installed DIRAC v8.0.50, based on DISET's CS. Meanwhile, I also want to add the TokenManager module (which only supports HTTPs). Therefore, I added the following line in the install.cfg file:

Services += Framework/TokenManager

After completing the installation, I configured the /opt/dirac/etc/CAS_Prod.cfg file as follows:

Systems
{
  Tornado
  {
    Production
    {
      Port = 8443
    }
  }
...
  Framework
  {
    Production
    {
      Services
      {
...
        TokenManager
        {
          Protocol = https
          HandlerPath = DIRAC/FrameworkSystem/Service/TokenManagerHandler.py
          Authorization
          {
            #Settings by default:
            Default = authenticated
            getUsersTokensInfo = ProxyManagement
          }
        }
    }
  }
}

However, I am encountering the following error in the log:

2024-08-30 03:26:54 UTC Framework/TokenManager ERROR: There were errors when loading configuration
2024-08-30 03:26:55 UTC Framework/TokenManager FATAL: Missing mandatory local configuration option /Systems/Framework/Production/Services/TokenManager/Port

However, the manual mentions that the port definition should be removed from CS. What is happening?

Additionally, when I tried to add port = 8443 in the TokenManager configuration, the output changed to:

2024-08-30 05:34:17 UTC Framework/TokenManager INFO: Loading Framework/TokenManager
2024-08-30 05:34:17 UTC Framework/TokenManager INFO: Trying to load Framework/TokenManager from CS defined path DIRAC/FrameworkSystem/Service/TokenManagerHandler.py
2024-08-30 05:34:17 UTC Framework/TokenManager ERROR: Framework/TokenManager has to inherit from RequestHandler

[aldbr]

Hello,

I have installed DIRAC 8.0.5

Dirac v8.0.5 or v8.0.50?
If it is v8.0.5, I would highly suggest you to update your Dirac instance to the latest v8.0 version (currently v8.0.50) as the Token Manager was still experimental at this stage.

[hanx-hep]

Version is v8.0.50. Sorry for the inconvenience.

[fstagni]

First, congratulations on your (not easy) effort while navigating the DIRAC configuration.

Regarding your issue: when a DIRAC service is run with https, the handler would be Tornado, meaning that you need to carefully follow the installation instructions at https://dirac.readthedocs.io/en/latest/AdministratorGuide/ServerInstallations/HTTPSServices.html and specifically (which I think it is what you are missing) run dirac-install-tornado-service.

Then, you will be able to follow the logs in Tornado/Tornado and not in Framework/TokenManager.

[hanx-hep]

Hi Stagni @fstagni , thanks for your help. I did miss the command dirac-install-tornado-service, but I encountered an error when I ran it:

dirac$ source /opt/dirac/bashrc 
dirac$ dirac-install-tornado-service 

I ran the above command trying to do the initial installation and configuration, but it returned:

2024-08-31 07:08:10 UTC Framework WARN: Using port '3306' as MySQL port
2024-08-31 07:08:10 UTC Framework WARN: Using 'root' as root MySQL user
2024-08-31 07:08:10 UTC Framework WARN: Using the same host for NoSQL as dirac services
2024-08-31 07:08:10 UTC Framework WARN: Using the default port 9200
2024-08-31 07:08:10 UTC Framework WARN: NoSQL user not found
2024-08-31 07:08:10 UTC Framework WARN: NoSQL password not found
2024-08-31 07:08:10 UTC Framework WARN: NoSQL SSL choice not found
No proxy found
Can't update from server Error while updating from dips://dirac02.ihep.ac.cn:9135/Configuration/Server: No proxy found
No proxy found
Can't update from server Error while updating from dips://dirac02.ihep.ac.cn:9135/Configuration/Server: No proxy found
No proxy found!
{'OK': False, 'Errno': 0, 'Message': 'Cannot locate client credentials', 'CallStack': ['  File "/opt/dirac/versions/v8.0.50-1724810378/Linux-x86_64/bin/dirac-install-tornado-service", line 8, in <module>\n    sys.exit(main())\n', '  File "/opt/dirac/versions/v8.0.50-1724810378/Linux-x86_64/lib/python3.11/site-packages/DIRAC/Core/Base/Script.py", line 74, in __call__\n    return entrypointFunc._func()\n', '  File "/opt/dirac/versions/v8.0.50-1724810378/Linux-x86_64/lib/python3.11/site-packages/DIRAC/FrameworkSystem/scripts/dirac_install_tornado_service.py", line 92, in main\n    result = gComponentInstaller.addTornadoOptionsToCS(gConfig)\n', '  File "/opt/dirac/versions/v8.0.50-1724810378/Linux-x86_64/lib/python3.11/site-packages/DIRAC/FrameworkSystem/Client/ComponentInstaller.py", line 2411, in addTornadoOptionsToCS\n    return self._addCfgToCS(cfg)\n', '  File "/opt/dirac/versions/v8.0.50-1724810378/Linux-x86_64/lib/python3.11/site-packages/DIRAC/FrameworkSystem/Client/ComponentInstaller.py", line 349, in _addCfgToCS\n    cfgClient = CSAPI()\n', '  File "/opt/dirac/versions/v8.0.50-1724810378/Linux-x86_64/lib/python3.11/site-packages/DIRAC/ConfigurationSystem/Client/CSAPI.py", line 37, in __init__\n    self.initialize()\n', '  File "/opt/dirac/versions/v8.0.50-1724810378/Linux-x86_64/lib/python3.11/site-packages/DIRAC/ConfigurationSystem/Client/CSAPI.py", line 97, in initialize\n    self.__initialized = S_ERROR("Cannot locate client credentials")\n']}
CSAPI not yet initialized

Does this seem to be a CS configuration issue?
Do I need to add the proxy configuration in the Configuration/Server?

[fstagni]

Sorry for the late answer, I was on holiday and AFK. The error message means that you do not have a proxy. You can anyway use the local certificate by using:

dirac-install-tornado-service --UseServerCertificate -o /DIRAC/Security/CertFile=%s/hostcert.pem -o /DIRAC/Security/KeyFile=%s/hostkey.pem

(put the right locations)

[hanx-hep]

Hi Fstagni, thank you for your help again. However, it seems that the --UseServerCertificate option does not exist in dirac-install-tornado-service:

$ dirac-install-tornado-service --UseServerCertificate -o /DIRAC/Security/CertFile=/opt/dirac/etc/grid-security/hostcert.pem -o /DIRAC/Security/KeyFile=/opt/dirac/etc/grid-security/hostkey.pem            130 ↵
2024-09-18 05:26:44 UTC Framework WARN: Using port '3306' as MySQL port
2024-09-18 05:26:44 UTC Framework WARN: Using 'root' as root MySQL user
2024-09-18 05:26:44 UTC Framework WARN: Using the same host for NoSQL as dirac services
2024-09-18 05:26:44 UTC Framework WARN: Using the default port 9200
2024-09-18 05:26:44 UTC Framework WARN: NoSQL user not found
2024-09-18 05:26:44 UTC Framework WARN: NoSQL password not found
2024-09-18 05:26:44 UTC Framework WARN: NoSQL SSL choice not found
Error when parsing command line arguments: option --UseServerCertificate not recognized

Do the initial installation and configuration of a DIRAC service based on tornado. If the component is not specified, just install Tornado as such (note that the installation will fail if there are no suited services in the CS)

[hanx-hep]

Of course, my configuration(dirac.cfg) is as follows, and I installed it using the command ./install_site.sh -i /opt/dirac -p WebAppDIRAC -p DIRACWebAppResources -p IHEPDIRAC -p SAMTest /home/dirac/DIRAC/dirac.cfg.

LocalInstallation
{
  ConfigurationServer = dips://diracx02.ihep.ac.cn:9135/Configuration/Server
  InstallType = server
  AdminUserDN = /C=CN/O=HEP/O=IHEP/OU=CC/CN=Xiao Han
  SkipCADonwload = yes
  RootPath = /opt/dirac/pro
  AdminUserName = hanx
  UseVersionsDir = True
  WebApp = yes
  AdminUserEmail = [email protected]
  ConfigurationMaster = yes
  Executors = WorkloadManagement/Optimizers
  SiteName = ihep.ac.cn
  UseServerCertificate = True
  Host = diracx.ihep.ac.cn
  AdminGroupName = dirac_admin
  ConfigurationName = CAS_Prod
  Services  = Configuration/Server
  Services += Framework/ComponentMonitoring
  Services += Framework/ProxyManager
  Services += Framework/UserProfileManager
  Services += Framework/SystemAdministrator
  Services += Framework/TokenManager
  Services += Framework/BundleDelivery
  Services += Framework/Notification
  Services += Framework/SecurityLogging
  Services += DataManagement/FTS3Manager
  Services += DataManagement/StorageElement
  Services += DataManagement/FileCatalog
  Services += DataManagement/StorageElementProxy
  Services += WorkloadManagement/SandboxStore
  Services += WorkloadManagement/Matcher
  Services += WorkloadManagement/JobMonitoring
  Services += WorkloadManagement/JobManager
  Services += WorkloadManagement/JobStateUpdate
  Services += WorkloadManagement/WMSAdministrator
  Services += WorkloadManagement/OptimizationMind
  Services += WorkloadManagement/PilotManager
  Services += WorkloadManagement/TaskManager
  Services += RequestManagement/ReqManager
  Services += RequestManagement/ReqProxy
  Services += Accounting/DataStore
  Services += Accounting/ReportGenerator
  Services += ResourceStatus/ResourceStatus
  Services += ResourceStatus/ResourceManagement
  Services += ResourceStatus/Publisher
  Services += Monitoring/Monitoring
  Services += Transformation/TransformationManager
  WebPortal = yes
  WebApp = yes
  InstancePath = /opt/dirac
  Setup = CAS_Production
  Project = IHEP
  Extensions = WebAppDIRAC, VMDIRAC
  ExtraModules = WebAppDIRAC, VMDIRAC
  SkipCAChecks = no
  Databases  = AccountingDB
  Databases += SandboxMetadataDB
  Databases += JobDB
  Databases += FileCatalogDB
  Databases += JobLoggingDB
  Databases += UserProfileDB
  Databases += TaskQueueDB
  Databases += NotificationDB
  Databases += InstalledComponentsDB
  Databases += ProxyDB
  Databases += PilotAgentsDB
  Databases += FTS3DB
  Databases += ReqDB
  Databases += ResourceManagementDB
  Databases += ResourceStatusDB
  Databases += TransformationDB
  Release = v0r1
  LcgVer = v14r1
  InstanceName = Production
  InstallMySQL = False
  Agents  = Framework/ComponentSupervisionAgent
  Agents += Framework/TopErrorMessagesReporter
  Agents += Framework/SAMLauncherAgent
  Agents += Framework/SAMLauncherAgentabc
  Agents += WorkloadManagement/PilotSyncAgent
  Agents += WorkloadManagement/PilotStatusAgent
  Agents += WorkloadManagement/SiteDirector
  Agents += WorkloadManagement/InputDataAgent
  Agents += WorkloadManagement/JobCleaningAgent
  Agents += WorkloadManagement/StalledJobAgent
  Agents += WorkloadManagement/StatesAccountingAgent
  Agents += WorkloadManagement/TaskQueuesAgent
  Agents += WorkloadManagement/SiteDirector
  Agents += WorkloadManagement/PilotMonitorAgent
  Agents += DataManagement/FTS3Agent
  Agents += RequestManagement/RequestExecutingAgent
  Agents += Configuration/VOMS2CSAgent
  Agents += ResourceStatus/CacheFeederAgent
  Agents += ResourceStatus/SAMTestAgent
  Agents += ResourceStatus/EmailAgent
  Agents += ResourceStatus/SiteInspectorAgent
  Agents += Transformation/InputDataAgent
  Agents += Transformation/MCExtensionAgent
  Agents += Transformation/RequestTaskAgent
  Agents += Transformation/TransformationAgent
  Agents += Transformation/TransformationCleaningAgent
  Agents += Transformation/ValidateOutputDataAgent
  Agents += Transformation/WorkflowTaskAgent
  Agents += WorkloadManagement/StatesMonitoringAgent
  HostDN = /C=CN/O=HEP/O=IHEP/OU=CC/CN=diracx02.ihep.ac.cn
  TargetPath = /opt/dirac
  Database
  {
    Host = localhost
    Password = XXXXXX
    User = Dirac
    RootPwd = XXXXXX
  }
  NoSQLDatabase
  {
    Host =
    Port =
    User =
    Password =
    SSL =
  }
}
DIRAC
{
  Setup = CAS_Production
  Hostname = diracx02.ihep.ac.cn
  Setups
  {
    CAS_Production
    {
      ResourceStatus = Production
      DataManagement = Production
      Framework = Production
      WorkloadManagement = Production
      RequestManagement = Production
      Transformation = Production
      Accounting = Production
      Monitoring = Production
      Configuration = Production
      Tornado = Production
    }
  }
  Security
  {

  }
  Configuration
  {
    Master = yes
    Name = CAS_Prod
    Servers = dips://diracx02.ihep.ac.cn:9135/Configuration/Server
  }
}
Systems
{
  Tornado
  {
    Production
    {
      Port = 8443
    }
  }
  Databases
  {
    Host = localhost
    Password = XXXXX
    User = Dirac
    RootPwd = XXXXX
  }
  NoSQLDatabase
  {
    Host =
    Port =
    User =
    Password =
    SSL =
  }
}

for dirac-service Framework/TokenManager -ddd, the result is show below:

(base) $ dirac-service Framework/TokenManager -ddd
2024-09-26 14:17:24 UTC Framework [139667278034752] DEBUG: dirac.cfg should be at /opt/dirac/etc/dirac.cfg
2024-09-26 14:17:24 UTC Framework [139667278034752] DEBUG: CFG merged
2024-09-26 14:17:24 UTC Framework [139667278034752] DEBUG: Updating configuration internals
2024-09-26 14:17:24 UTC Framework [139667278034752] DEBUG: Updating configuration internals
2024-09-26 14:17:24 UTC Framework [139667278034752] DEBUG: Parsing command line
2024-09-26 14:17:24 UTC Framework [139667278034752] DEBUG: Updating configuration internals
2024-09-26 14:17:24 UTC Framework [139667278034752] DEBUG: Refreshing configuration...
2024-09-26 14:17:24 UTC Framework [139667278034752] DEBUG: Refreshing from list ['dips://diracx02.ihep.ac.cn:9135/Configuration/Server']
2024-09-26 14:17:24 UTC Framework [139667278034752] DEBUG: Randomized server list is dips://diracx02.ihep.ac.cn:9135/Configuration/Server
2024-09-26 14:17:24 UTC Framework [139667278034752] DEBUG:  Trying to refresh from dips://diracx02.ihep.ac.cn:9135/Configuration/Server
2024-09-26 14:17:24 UTC Framework/DIRAC.Core.Tornado.Client.ClientSelector [139667278034752] DEBUG: Trying to autodetect client for dips://diracx02.ihep.ac.cn:9135/Configuration/Server
2024-09-26 14:17:24 UTC Framework [139667278034752] DEBUG: Already given a valid url dips://diracx02.ihep.ac.cn:9135/Configuration/Server
2024-09-26 14:17:24 UTC Framework [139667278034752] DEBUG: Trying to connect to: dips://diracx02.ihep.ac.cn:9135/Configuration/Server
2024-09-26 14:17:24 UTC Framework [139667278034752] DEBUG: Connected to: dips://diracx02.ihep.ac.cn:9135/Configuration/Server
2024-09-26 14:17:24 UTC Framework [139667278034752] DEBUG: New connection -> 202.122.33.154:9135
2024-09-26 14:17:24 UTC Framework [139667278034752] DEBUG: New version available Updating to version 2024-09-23 05:41:04.484021...
2024-09-26 14:17:24 UTC Framework [139667278034752] DEBUG: Updating configuration internals
2024-09-26 14:17:24 UTC Framework [139667278034752] DEBUG: Updated to version 2024-09-23 05:41:04.484021
2024-09-26 14:17:24 UTC Framework [139667278034752] DEBUG: Updating configuration internals
2024-09-26 14:17:24 UTC Framework/TokenManager [139667278034752] DEBUG: Trying to load Resources.LogBackends.StdoutBackend
2024-09-26 14:17:24 UTC Framework/TokenManager [139667278034752] DEBUG: Trying to load IHEPDIRAC.Resources.LogBackends.StdoutBackend
2024-09-26 14:17:24 UTC Framework/TokenManager [139667278034752] DEBUG: Trying to load WebAppDIRAC.Resources.LogBackends.StdoutBackend
2024-09-26 14:17:24 UTC Framework/TokenManager [139667278034752] DEBUG: Trying to load DIRACWebAppResources.Resources.LogBackends.StdoutBackend
2024-09-26 14:17:24 UTC Framework/TokenManager [139667278034752] DEBUG: Trying to load DIRAC.Resources.LogBackends.StdoutBackend
2024-09-26 14:17:24 UTC Framework/TokenManager [139667278034752] FATAL: Missing mandatory local configuration option /Systems/Framework/Production/Services/TokenManager/Port
2024-09-26 14:17:24 UTC Framework/TokenManager [139667278034752] ERROR: There were errors when loading configuration

[fstagni]

The configuration you posted is from the local dirac.cfg. In the same directory there's the full configuration (probably called Production.cfg) which is what you get also by downloading it from the WebApp:

[hanx-hep]

The configuration I posted earlier is the installation configuration file. I misunderstood your request. The whole configuration file is attached (called CAS_Prod.cfg /opt/dirac/etc/CAS_Prod.cfg) .
CAS_Prod.cfg.txt

[fstagni]

Config seems OK. What do you have in /opt/dirac/runit/Tornado/Tornado/run ?

It should be:

[fstagni@lbvobox316 dirac]$ cat runit/Tornado/Tornado/run
#!/bin/bash
rcfile=/opt/dirac/bashrc
[ -e $rcfile ] && source $rcfile
#
export DIRAC_USE_TORNADO_IOLOOP=Yes
exec 2>&1
#
#
exec tornado-start-all

[hanx-hep]

Yes, I have same script in /opt/dirac/runit/Tornado/Tornado/run.

By the way, should I install DIRAC first (without Services += Framework/TokenManager), then install TokenManager using dirac-install-tornado-service? For now, I install them together.