Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: deflake metrics test #420

Merged
merged 1 commit into from
Feb 5, 2024
Merged

chore: deflake metrics test #420

merged 1 commit into from
Feb 5, 2024

Conversation

jcosentino11
Copy link
Member

Issue #, if available:

Description of changes:

Attempt to get rid of null pointers showing up in recent test runs for GIVEN_kernelRunningWithMetricsConfig_WHEN_verifyClientDeviceIdentity_THEN_successMetricEmitted

Why is this change necessary:

How was this change tested:

Any additional information or context required to review the change:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

//Wait for the emitter to run
Mockito.verify(metricSpy, Mockito.timeout(2000L).atLeastOnce()).emitMetrics();
AtomicReference<List<Metric>> results = new AtomicReference<>();
eventuallyTrue(() -> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you need to assert on this eventually true

ResultCaptor<List<Metric>> resultMetrics = new ResultCaptor<>();
doAnswer(resultMetrics).when(metricSpy).collectMetrics();

//Wait for the emitter to run
Mockito.verify(metricSpy, Mockito.timeout(2000L).atLeastOnce()).emitMetrics();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

keeping the mockito verify should be used instead of eventually true. Was it the reset which was clearing things out?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah was thinking it's a race between setting up the mock and performing the ipc call. removed the eventually true and just re-ordered things

Copy link

github-actions bot commented Feb 5, 2024

Code Coverage Report

File Coverage
All files 74%
com.aws.greengrass.clientdevices.auth.PermissionEvaluationUtils 77%
com.aws.greengrass.clientdevices.auth.CertificateManager 88%
com.aws.greengrass.clientdevices.auth.ClientDevicesAuthService 79%
com.aws.greengrass.clientdevices.auth.DeviceAuthClient 73%
com.aws.greengrass.clientdevices.auth.certificate.ClientCertificateGenerator 95%
com.aws.greengrass.clientdevices.auth.certificate.CertificateHelper 74%
com.aws.greengrass.clientdevices.auth.certificate.CertificateStore 72%
com.aws.greengrass.clientdevices.auth.certificate.CertificateExpiryMonitor 77%
com.aws.greengrass.clientdevices.auth.certificate.ServerCertificateGenerator 93%
com.aws.greengrass.clientdevices.auth.certificate.CertificateGenerator 70%
com.aws.greengrass.clientdevices.auth.certificate.CertificateExpiryMonitor$CertRotationDecider 90%
com.aws.greengrass.clientdevices.auth.iot.usecases.VerifyIotCertificate 94%
com.aws.greengrass.clientdevices.auth.iot.usecases.VerifyThingAttachedToCertificate 89%
com.aws.greengrass.clientdevices.auth.iot.usecases.CreateIoTThingSession 67%
com.aws.greengrass.clientdevices.auth.iot.usecases.VerifyCertificateValidityPeriod 88%
com.aws.greengrass.clientdevices.auth.certificate.infra.BackgroundCertificateRefresh 82%
com.aws.greengrass.clientdevices.auth.iot.infra.ThingRegistry 92%
com.aws.greengrass.clientdevices.auth.certificate.usecases.ConfigureManagedCertificateAuthority 85%
com.aws.greengrass.clientdevices.auth.certificate.usecases.ConfigureCustomCertificateAuthority 83%
com.aws.greengrass.clientdevices.auth.certificate.usecases.RegisterCertificateAuthorityUseCase 65%
com.aws.greengrass.clientdevices.auth.configuration.MetricsConfiguration 83%
com.aws.greengrass.clientdevices.auth.configuration.GroupManager 89%
com.aws.greengrass.clientdevices.auth.configuration.CAConfiguration 96%
com.aws.greengrass.clientdevices.auth.configuration.RuntimeConfiguration 83%
com.aws.greengrass.clientdevices.auth.configuration.SecurityConfiguration 80%
com.aws.greengrass.clientdevices.auth.configuration.GroupDefinition 75%
com.aws.greengrass.clientdevices.auth.configuration.ExpressionVisitor 84%
com.aws.greengrass.clientdevices.auth.configuration.GroupConfiguration 92%
com.aws.greengrass.clientdevices.auth.api.ClientDevicesAuthServiceApi 85%
com.aws.greengrass.clientdevices.auth.api.GetCertificateRequestWithGenerator 75%
com.aws.greengrass.clientdevices.auth.api.UseCases 71%
com.aws.greengrass.clientdevices.auth.session.attribute.WildcardSuffixAttribute 96%
com.aws.greengrass.clientdevices.auth.iot.IotAuthClient$Default 56%
com.aws.greengrass.clientdevices.auth.iot.Thing 85%
com.aws.greengrass.clientdevices.auth.iot.Certificate 74%
com.aws.greengrass.clientdevices.auth.iot.GreengrassV2DataClientFactory 18%
com.aws.greengrass.clientdevices.auth.iot.CertificateRegistry 95%
com.aws.greengrass.clientdevices.auth.infra.NetworkStateProvider$Default 83%
com.aws.greengrass.ipc.IPCUtils 83%
com.aws.greengrass.ipc.VerifyClientDeviceIdentityOperationHandler 60%
com.aws.greengrass.ipc.GetClientDeviceAuthTokenOperationHandler 83%
com.aws.greengrass.ipc.AuthorizeClientDeviceActionOperationHandler 79%
com.aws.greengrass.ipc.SubscribeToCertificateUpdatesOperationHandler 81%
com.aws.greengrass.clientdevices.auth.session.SessionConfig 92%
com.aws.greengrass.clientdevices.auth.session.SessionManager 88%
com.aws.greengrass.clientdevices.auth.certificate.handlers.CAConfigurationChangedHandler 93%
com.aws.greengrass.clientdevices.auth.certificate.handlers.CertificateRotationHandler 96%
com.aws.greengrass.clientdevices.auth.metrics.handlers.SessionCreationEventHandler 88%
com.aws.greengrass.clientdevices.auth.metrics.handlers.MetricsConfigurationChangedHandler 70%
com.aws.greengrass.clientdevices.auth.metrics.handlers.AuthorizeClientDeviceActionsMetricHandler 88%
com.aws.greengrass.clientdevices.auth.metrics.handlers.VerifyClientDeviceIdentityEventHandler 88%
com.aws.greengrass.clientdevices.auth.metrics.handlers.CertificateSubscriptionEventHandler 83%
com.aws.greengrass.clientdevices.auth.util.ResizableLinkedBlockingQueue 90%
com.aws.greengrass.clientdevices.auth.util.ParseIPAddress 90%
com.aws.greengrass.clientdevices.auth.connectivity.CISShadowMonitor$SucceedOnceOperation 88%
com.aws.greengrass.clientdevices.auth.connectivity.CISShadowMonitor 79%
com.aws.greengrass.clientdevices.auth.connectivity.CISShadowMonitor$ProcessCISShadowTask 83%
com.aws.greengrass.clientdevices.auth.connectivity.CISShadowMonitor$CISShadowTaskQueue 90%
com.aws.greengrass.clientdevices.auth.connectivity.HostAddress 67%
com.aws.greengrass.clientdevices.auth.connectivity.CISShadowMonitor$CISShadowTaskExecutor 80%
com.aws.greengrass.clientdevices.auth.configuration.parser.TokenMgrError 22%
com.aws.greengrass.clientdevices.auth.configuration.parser.RuleExpressionTokenManager 61%
com.aws.greengrass.clientdevices.auth.configuration.parser.ASTStart 33%
com.aws.greengrass.clientdevices.auth.configuration.parser.ASTAnd 67%
com.aws.greengrass.clientdevices.auth.configuration.parser.Token 58%
com.aws.greengrass.clientdevices.auth.configuration.parser.RuleExpressionDefaultVisitor 0%
com.aws.greengrass.clientdevices.auth.configuration.parser.ASTOr 67%
com.aws.greengrass.clientdevices.auth.configuration.parser.SimpleCharStream 28%
com.aws.greengrass.clientdevices.auth.configuration.parser.RuleExpressionTreeConstants 0%
com.aws.greengrass.clientdevices.auth.configuration.parser.JJTRuleExpressionState 67%
com.aws.greengrass.clientdevices.auth.configuration.parser.ASTThing 67%
com.aws.greengrass.clientdevices.auth.configuration.parser.RuleExpression 63%
com.aws.greengrass.clientdevices.auth.configuration.parser.SimpleNode 27%

Minimum allowed coverage is 50%

Generated by 🐒 cobertura-action against 6b480ba

Copy link

github-actions bot commented Feb 5, 2024

Benchmark Results

Benchmark Score
com.aws.greengrass.clientdevices.auth.benchmark.AuthorizationBenchmarks.GIVEN_policy_with_thing_name_variable_WHEN_auth_request_THEN_successful_auth 756248.53 ops/s
com.aws.greengrass.clientdevices.auth.benchmark.AuthorizationBenchmarks.GIVEN_single_group_permission_WHEN_simple_auth_request_THEN_successful_auth 1068834.34 ops/s

@jcosentino11 jcosentino11 merged commit ce7453b into main Feb 5, 2024
5 checks passed
@jcosentino11 jcosentino11 deleted the deflake-metrics-test branch February 5, 2024 21:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants