Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(uat): implement scenario outline GGMQ-1-T27 #379

Merged
merged 9 commits into from
Jul 27, 2023
Merged

Conversation

bgklika
Copy link

@bgklika bgklika commented Jul 26, 2023

Issue #, if available:
Implement GGMQ scenario based on GGAD-1-T27

Description of changes:

  • Add GGMQ-1-T27 scenario outline
  • Move list of addresses of brokers from MqttControlSteps to MqttBrokers
  • Add BrokerCertificateSteps with implementation of certificate getting and comparision
  • In addition: update readme with description of tags

Why is this change necessary:
Implement replacement of GGAD-1-T27 scenario outline

How was this change tested:
Scenarios have been run on Linux for 4 clients (later reduced to 1) and Windows (one client) for both MQTT v3 and v5.

Test results:
Linux:

[INFO ] 2023-07-25 22:46:59.173 [main] StepTrackingReporting - Passed: 'GGMQ-1-T27-v3-sdk-java: As a customer, my Greengrass-issued certificate does not rotate when mqtt reconnects'
[INFO ] 2023-07-25 22:46:59.173 [main] StepTrackingReporting - Passed: 'GGMQ-1-T27-v5-sdk-java: As a customer, my Greengrass-issued certificate does not rotate when mqtt reconnects'

Windows:

[INFO ] 2023-07-25 23:22:47.231 [main] StepTrackingReporting - Passed: 'GGMQ-1-T27-v3-sdk-java: As a customer, my Greengrass-issued certificate does not rotate when mqtt reconnects'
[INFO ] 2023-07-25 23:22:47.231 [main] StepTrackingReporting - Passed: 'GGMQ-1-T27-v5-sdk-java: As a customer, my Greengrass-issued certificate does not rotate when mqtt reconnects'

Any additional information or context required to review the change:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@bgklika bgklika requested a review from MikeDombo July 26, 2023 10:36
@github-actions
Copy link

github-actions bot commented Jul 26, 2023

Unit Tests Coverage Report

File Coverage Lines Branches
All files 72% 79% 65%
com.aws.greengrass.clientdevices.auth.PermissionEvaluationUtils 78% 82% 74%
com.aws.greengrass.clientdevices.auth.PermissionEvaluationUtils$Operation 100% 100% 0%
com.aws.greengrass.clientdevices.auth.PermissionEvaluationUtils$Resource 100% 100% 0%
com.aws.greengrass.clientdevices.auth.CertificateManager 79% 90% 69%
com.aws.greengrass.clientdevices.auth.ClientDevicesAuthService 78% 90% 67%
com.aws.greengrass.clientdevices.auth.DeviceAuthClient 73% 83% 64%
com.aws.greengrass.clientdevices.auth.certificate.ClientCertificateGenerator 95% 90% 100%
com.aws.greengrass.clientdevices.auth.certificate.CertificateHelper 74% 93% 54%
com.aws.greengrass.clientdevices.auth.certificate.CertificateHelper$ProviderType 100% 100% 0%
com.aws.greengrass.clientdevices.auth.certificate.CertificateStore 72% 85% 60%
com.aws.greengrass.clientdevices.auth.certificate.CertificateExpiryMonitor 77% 87% 67%
com.aws.greengrass.clientdevices.auth.certificate.ServerCertificateGenerator 93% 87% 100%
com.aws.greengrass.clientdevices.auth.certificate.CertificateGenerator 70% 90% 50%
com.aws.greengrass.clientdevices.auth.certificate.CertificateStore$CAType 100% 100% 0%
com.aws.greengrass.clientdevices.auth.certificate.CertificateExpiryMonitor$CertRotationDecider 90% 100% 80%
com.aws.greengrass.clientdevices.auth.certificate.CertificatesConfig 100% 100% 100%
com.aws.greengrass.clientdevices.auth.session.events.SessionCreationEvent$SessionCreationStatus 100% 100% 0%
com.aws.greengrass.clientdevices.auth.iot.usecases.VerifyIotCertificate 94% 88% 100%
com.aws.greengrass.clientdevices.auth.iot.usecases.VerifyThingAttachedToCertificate 92% 96% 88%
com.aws.greengrass.clientdevices.auth.iot.usecases.CreateIoTThingSession 87% 90% 83%
com.aws.greengrass.clientdevices.auth.iot.usecases.VerifyCertificateValidityPeriod 88% 88% 0%
com.aws.greengrass.clientdevices.auth.certificate.infra.ClientCertificateStore 100% 100% 100%
com.aws.greengrass.clientdevices.auth.certificate.infra.BackgroundCertificateRefresh 83% 85% 82%
com.aws.greengrass.clientdevices.auth.iot.infra.ThingRegistry 92% 97% 88%
com.aws.greengrass.clientdevices.auth.certificate.usecases.ConfigureManagedCertificateAuthority 85% 85% 0%
com.aws.greengrass.clientdevices.auth.certificate.usecases.ConfigureCustomCertificateAuthority 83% 83% 0%
com.aws.greengrass.clientdevices.auth.certificate.usecases.RegisterCertificateAuthorityUseCase 65% 81% 50%
com.aws.greengrass.clientdevices.auth.configuration.MetricsConfiguration 83% 100% 67%
com.aws.greengrass.clientdevices.auth.configuration.AuthorizationPolicyStatement$Effect 100% 100% 0%
com.aws.greengrass.clientdevices.auth.configuration.GroupManager 89% 94% 83%
com.aws.greengrass.clientdevices.auth.configuration.ConfigurationFormatVersion 100% 100% 0%
com.aws.greengrass.clientdevices.auth.configuration.CAConfiguration 96% 100% 92%
com.aws.greengrass.clientdevices.auth.configuration.RuntimeConfiguration 84% 99% 70%
com.aws.greengrass.clientdevices.auth.configuration.SecurityConfiguration 80% 93% 67%
com.aws.greengrass.clientdevices.auth.configuration.CDAConfiguration 100% 100% 100%
com.aws.greengrass.clientdevices.auth.configuration.GroupDefinition 75% 100% 50%
com.aws.greengrass.clientdevices.auth.configuration.ExpressionVisitor 84% 94% 75%
com.aws.greengrass.clientdevices.auth.configuration.GroupConfiguration 90% 95% 86%
com.aws.greengrass.clientdevices.auth.api.ServiceErrorEvent 100% 100% 0%
com.aws.greengrass.clientdevices.auth.api.ClientDevicesAuthServiceApi 90% 79% 100%
com.aws.greengrass.clientdevices.auth.api.DomainEvents 100% 100% 100%
com.aws.greengrass.clientdevices.auth.api.AuthorizeClientDeviceActionEvent$AuthorizationStatus 100% 100% 0%
com.aws.greengrass.clientdevices.auth.api.UseCases 71% 92% 50%
com.aws.greengrass.clientdevices.auth.api.DomainEvent 100% 100% 0%
com.aws.greengrass.clientdevices.auth.api.GetCertificateRequestOptions$CertificateType 100% 100% 0%
com.aws.greengrass.clientdevices.auth.session.attribute.StringLiteralAttribute 100% 100% 0%
com.aws.greengrass.clientdevices.auth.session.attribute.WildcardSuffixAttribute 88% 100% 75%
com.aws.greengrass.clientdevices.auth.certificate.events.CertificateSubscriptionEvent$SubscriptionStatus 100% 100% 0%
com.aws.greengrass.clientdevices.auth.certificate.events.CACertificateChainChanged 100% 100% 0%
com.aws.greengrass.clientdevices.auth.iot.Certificate$Status 100% 100% 0%
com.aws.greengrass.clientdevices.auth.iot.IotAuthClient$Default 56% 47% 64%
com.aws.greengrass.clientdevices.auth.iot.Thing 82% 86% 79%
com.aws.greengrass.clientdevices.auth.iot.Certificate 78% 89% 67%
com.aws.greengrass.clientdevices.auth.iot.GreengrassV2DataClientFactory 18% 18% 0%
com.aws.greengrass.clientdevices.auth.iot.CertificateRegistry 95% 90% 100%
com.aws.greengrass.clientdevices.auth.iot.Component 100% 100% 0%
com.aws.greengrass.clientdevices.auth.iot.events.VerifyClientDeviceIdentityEvent$VerificationStatus 100% 100% 0%
com.aws.greengrass.clientdevices.auth.infra.NetworkStateProvider$Default$1 100% 100% 0%
com.aws.greengrass.clientdevices.auth.infra.NetworkStateProvider$ConnectionState 100% 100% 0%
com.aws.greengrass.clientdevices.auth.infra.NetworkStateProvider$Default 75% 90% 60%
com.aws.greengrass.ipc.IPCUtils 83% 67% 100%
com.aws.greengrass.ipc.VerifyClientDeviceIdentityOperationHandler 60% 69% 50%
com.aws.greengrass.ipc.GetClientDeviceAuthTokenOperationHandler 86% 98% 75%
com.aws.greengrass.ipc.AuthorizeClientDeviceActionOperationHandler 79% 92% 67%
com.aws.greengrass.ipc.SubscribeToCertificateUpdatesOperationHandler 81% 88% 75%
com.aws.greengrass.clientdevices.auth.session.SessionConfig 92% 100% 83%
com.aws.greengrass.clientdevices.auth.session.SessionManager$1 100% 100% 100%
com.aws.greengrass.clientdevices.auth.session.MqttSessionFactory 100% 100% 100%
com.aws.greengrass.clientdevices.auth.session.SessionCreator 100% 100% 100%
com.aws.greengrass.clientdevices.auth.session.SessionManager 88% 100% 75%
com.aws.greengrass.clientdevices.auth.session.SessionImpl 100% 100% 100%
com.aws.greengrass.clientdevices.auth.session.SessionCreator$SessionFactorySingleton 100% 100% 0%
com.aws.greengrass.clientdevices.auth.session.MqttSessionFactory$MqttCredential 100% 100% 0%
com.aws.greengrass.clientdevices.auth.certificate.handlers.CACertificateChainChangedHandler 100% 100% 0%
com.aws.greengrass.clientdevices.auth.certificate.handlers.CAConfigurationChangedHandler 93% 87% 100%
com.aws.greengrass.clientdevices.auth.certificate.handlers.CertificateRotationHandler 96% 91% 100%
com.aws.greengrass.clientdevices.auth.certificate.handlers.SecurityConfigurationChangedHandler 100% 100% 0%
com.aws.greengrass.clientdevices.auth.metrics.handlers.SessionCreationEventHandler 88% 100% 75%
com.aws.greengrass.clientdevices.auth.metrics.handlers.MetricsConfigurationChangedHandler 70% 90% 50%
com.aws.greengrass.clientdevices.auth.metrics.handlers.AuthorizeClientDeviceActionsMetricHandler 88% 100% 75%
com.aws.greengrass.clientdevices.auth.metrics.handlers.VerifyClientDeviceIdentityEventHandler 88% 100% 75%
com.aws.greengrass.clientdevices.auth.metrics.handlers.CertificateSubscriptionEventHandler 83% 100% 67%
com.aws.greengrass.clientdevices.auth.metrics.handlers.ServiceErrorEventHandler 100% 100% 0%
com.aws.greengrass.clientdevices.auth.iot.dto.CertificateV1DTO$Status 100% 100% 0%
com.aws.greengrass.clientdevices.auth.connectivity.usecases.GetConnectivityInformationUseCase 100% 100% 0%
com.aws.greengrass.clientdevices.auth.connectivity.usecases.RecordConnectivityChangesUseCase 100% 100% 100%
com.aws.greengrass.clientdevices.auth.util.ResizableLinkedBlockingQueue 90% 80% 100%
com.aws.greengrass.clientdevices.auth.util.ParseIPAddress 90% 95% 84%
com.aws.greengrass.clientdevices.auth.metrics.ClientDeviceAuthMetrics 87% 98% 75%
com.aws.greengrass.clientdevices.auth.metrics.MetricsEmitter 100% 100% 100%
com.aws.greengrass.clientdevices.auth.connectivity.CISShadowMonitor 63% 77% 50%
com.aws.greengrass.clientdevices.auth.connectivity.RecordConnectivityChangesResponse 100% 100% 100%
com.aws.greengrass.clientdevices.auth.connectivity.HostAddress 67% 67% 0%
com.aws.greengrass.clientdevices.auth.connectivity.RecordConnectivityChangesRequest 100% 100% 0%
com.aws.greengrass.clientdevices.auth.connectivity.ConnectivityInformation 100% 100% 100%
com.aws.greengrass.clientdevices.auth.configuration.parser.RuleExpressionConstants 100% 100% 0%
com.aws.greengrass.clientdevices.auth.configuration.parser.TokenMgrError 22% 32% 12%
com.aws.greengrass.clientdevices.auth.configuration.parser.RuleExpressionTokenManager 61% 65% 58%
com.aws.greengrass.clientdevices.auth.configuration.parser.ASTStart 33% 33% 0%
com.aws.greengrass.clientdevices.auth.configuration.parser.ASTAnd 67% 67% 0%
com.aws.greengrass.clientdevices.auth.configuration.parser.Token 58% 58% 0%
com.aws.greengrass.clientdevices.auth.configuration.parser.RuleExpressionDefaultVisitor 0% 0% 0%
com.aws.greengrass.clientdevices.auth.configuration.parser.ASTOr 67% 67% 0%
com.aws.greengrass.clientdevices.auth.configuration.parser.SimpleCharStream 28% 31% 25%
com.aws.greengrass.clientdevices.auth.configuration.parser.RuleExpressionTreeConstants 0% 0% 0%
com.aws.greengrass.clientdevices.auth.configuration.parser.JJTRuleExpressionState 67% 65% 70%
com.aws.greengrass.clientdevices.auth.configuration.parser.ASTThing 67% 67% 0%
com.aws.greengrass.clientdevices.auth.configuration.parser.RuleExpression 63% 63% 62%
com.aws.greengrass.clientdevices.auth.configuration.parser.SimpleNode 27% 35% 19%

Minimum allowed coverage is 50%

Generated by 🐒 cobertura-action against 130278a

Then I retrieve the certificate of broker "default_broker" and store as "BROKER_CERTIFICATE_BEFORE_DISCONNECT"

When I set device mqtt connectivity to offline
And the greengrass log on the device contains the line "com.aws.greengrass.mqttclient.AwsIotMqtt5Client: Failed to connect to AWS IoT Core" at least 3 times within 2 minutes
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do you want this 3 times? Why isn't 1 enough?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At first time 1 is enough. Usually Nucleus makes 3 attempts in the short period of time like:

2023-07-25T20:43:30.342Z [ERROR] (Thread-10) com.aws.greengrass.mqttclient.AwsIotMqtt5Client: Failed to connect to AWS IoT Core. {clientId=gg-5e37c73c40370bb47008-ggc-thing, error=socket operation timed out.}
2023-07-25T20:43:34.858Z [ERROR] (Thread-10) com.aws.greengrass.mqttclient.AwsIotMqtt5Client: Failed to connect to AWS IoT Core. {clientId=gg-5e37c73c40370bb47008-ggc-thing, error=socket operation timed out.}
2023-07-25T20:43:37.958Z [ERROR] (Thread-10) com.aws.greengrass.mqttclient.AwsIotMqtt5Client: Failed to connect to AWS IoT Core. {clientId=gg-5e37c73c40370bb47008-ggc-thing, error=socket operation timed out.}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So if you only need 1, why does this step require 3?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assumed Nucleus made series of attempts before falling to disconnected state.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, will change to 1

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

onDisconnection() log another message - "Connection interrupted".
Message "Failed to connect to AWS IoT Core" we are waiting for logged from onConnectionFailure() which called 3 times after onDisconnection() .

But "Connection interrupted" is appeared already in log file.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, "Connection interrupted" means that the device was disconnected. If you see this, and there is no "Connection resumed" then it is offline.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 224fb32

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At the moment has no steps to check log file miss some string.

@MikeDombo MikeDombo merged commit 4654f3d into uat-dev Jul 27, 2023
8 checks passed
@MikeDombo MikeDombo deleted the GGMQ-208-T27 branch July 27, 2023 23:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants