Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HDDS-10855. Handle Null ParentKeyInfo Error in Recon Namespace Summary Task. #6679

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

ArafatKhan2198
Copy link
Contributor

@ArafatKhan2198 ArafatKhan2198 commented May 15, 2024

What changes were proposed in this pull request?

This pull request fixes the Null ParentKeyInfo error that occurs during the Namespace Summary task in Recon. The error happens when the parent key information is missing. The update adds better logging and skips the creation of NSSummary for keys with missing parent info, so the task can continue processing other keys without stopping.

The error was causing the task to fail whenever it couldn't find the parent directory information. This could happen if the parent was deleted or not yet added to the database. By skipping these problematic keys and logging a warning, the task can keep running smoothly.

Approach:

  • Added detailed logging when the parent key info is missing.
  • Modified methods to throw an exception if the parent info is not found.
  • Updated processing methods to catch this exception, log it, and skip the creation of NSSummary for those keys.
  • Modified the handleDeleteKeyEvent to pick up the parent ID from the NSSummary object.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-10855

How was this patch tested?

Copy link
Contributor

@devmadhuu devmadhuu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ArafatKhan2198 for working on this patch. Have some comments. Pls check.

@@ -155,7 +155,8 @@ protected void handlePutDirEvent(OmDirectoryInfo directoryInfo,
protected void handleDeleteKeyEvent(OmKeyInfo keyInfo,
Map<Long, NSSummary> nsSummaryMap)
throws IOException {
long parentObjectId = keyInfo.getParentObjectID();
NSSummary currentNSSummary = nsSummaryMap.get(keyInfo.getObjectID());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not very clear why we have taken parent Id from NSSummary object based. What was the issue in earlier code if we directly take parentObjectId from OmKeyInfo here ?

try {
setParentBucketId(updatedKeyInfo);
} catch (IOException e) {
LOG.warn("Skipping NSSummary creation for key: {} due to missing parent bucket info. Exception: {}",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same warn message we are logging in setParentBucketId method also. processWithObjectStoreLayout is the caller of setParentBucketId, can we log different message which can mention what that caller is doing.

setKeyParentID(updatedKeyInfo);

try {
setKeyParentID(updatedKeyInfo);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same warn message we are logging in setKeyParentID method also. processWithFileSystemLayout is the caller of setKeyParentID, can we log different message which can mention what that caller is doing.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I couldn't find any logging in the setKeyParentID method, only the IOException is there.

private void setKeyParentID(OmKeyInfo keyInfo) throws IOException {
String[] keyPath = keyInfo.getKeyName().split(OM_KEY_PREFIX);
// If the path contains only one key then keyPath.length
// will be 1 and the parent will be a bucket.
// If the keyPath.length is greater than 1 then
// there is at least one directory.
if (keyPath.length > 1) {
String[] dirs = Arrays.copyOf(keyPath, keyPath.length - 1);
String parentKeyName = String.join(OM_KEY_PREFIX, dirs);
parentKeyName += OM_KEY_PREFIX;
String fullParentKeyName =
getReconOMMetadataManager().getOzoneKey(keyInfo.getVolumeName(),
keyInfo.getBucketName(), parentKeyName);
OmKeyInfo parentKeyInfo = getReconOMMetadataManager()
.getKeyTable(LEGACY_BUCKET_LAYOUT)
.getSkipCache(fullParentKeyName);
if (parentKeyInfo != null) {
keyInfo.setParentObjectID(parentKeyInfo.getObjectID());
} else {
throw new IOException("ParentKeyInfo for " +
"NSSummaryTaskWithLegacy is null");
}
} else {
setParentBucketId(keyInfo);
}
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
4 participants