Skip to content

sync: import latest source/taos-community subtree (3.3.6)#35381

Open
feici02 wants to merge 58 commits into
3.3.6from
sync/taos-community-3.3.6-20260606-194604
Open

sync: import latest source/taos-community subtree (3.3.6)#35381
feici02 wants to merge 58 commits into
3.3.6from
sync/taos-community-3.3.6-20260606-194604

Conversation

@feici02

@feici02 feici02 commented Jun 6, 2026

Copy link
Copy Markdown
Member

Sync source/taos-community/ subtree from the internal 3.3.6 monorepo branch.

  • Source: rd-public/tsdb branch 3.3.6 (HEAD: 4958eece10f)
  • Prefix: source/taos-community
  • Target: taosdata/tdengine branch 3.3.6
  • Traversal: --first-parent (each commit is the integration point of an MR; trees stay correct across parallel merges)

Generated by tools/sync/subtree-split-fast.sh with author/committer identities preserved on every commit.

guanshengliang and others added 30 commits March 30, 2026 16:09
…b8510fc1e197f'

git-subtree-dir: source/taos-community
git-subtree-mainline: 7fa2721
git-subtree-split: b94adf0
chore: sync from github

See merge request rd-public/tsdb!7
chore: build 3.3.6 in mac and linux platform

See merge request rd-public/tsdb!8
chore: build 3.3.6 in mac

See merge request rd-public/tsdb!9
chore: fix link warnings in mac

See merge request rd-public/tsdb!10
chore: build in mac

See merge request rd-public/tsdb!11
chore: sync from github 3.3.6

See merge request rd-public/tsdb!13
fix: resolve BUILD_TEST=ON compilation errors

See merge request rd-public/tsdb!25
chore: update cmake options

See merge request rd-public/tsdb!29
chore: update BUILD_CONTRIB and add BUILD_ROCKSDB

See merge request rd-public/tsdb!39
fix(taos-gen): 修复 conan 构建两类失败:过期 cmake 文件 & 跨镜像 ABI 不匹配

See merge request rd-public/tsdb!58
feat: support GitLab monorepo build - taosx cmake split and Windows iconv fix

See merge request rd-public/tsdb!49
fix: GCC 7 (core:0.2 / manylinux2014) 构建兼容性修复及 RocksDB 选项重构

See merge request rd-public/tsdb!76
….3.6'

fix(cmake): Fix rocksdb cache validation false failure caused by unparsed generator expression when BUILD_CONTRIB=OFF

See merge request rd-public/tsdb!86
chore: sync from github

See merge request rd-public/tsdb!131
chore: sync from github

See merge request rd-public/tsdb!163
ci: prepare to merge 3.3.6 for new ci

See merge request rd-public/tsdb!166
boxiaobj and others added 24 commits May 26, 2026 09:19
…b!410)

fix(3.3.6): port CI test failure fixes from main branch
…public/tsdb!450)

build(deps): add CPython Win/Mac tarballs and sync externals manifest [cherry-pick → 3.3.6]
refa: reorganize tsdb test pipeline into tools/cicd/tsdb-test-pipeline (3.3.6)
…blic/tsdb!455)

build: update x86 rocksdb prebuilt binary for gcc 11.2.1 compatibility
fix:(stmt)core when null string convert to ts
…!471)

fix(keeper): handle nil metric write responses
fix(parser/insert): align tag name array with cid-sorted tag values
feat: WAL corruption auto-recovery with backup support
…lic/tsdb!519)

fix(build): remove --strip-debug from Release linker flags
…349)

fix: two issues in copy mode repair - 3.3.6
feat(build): automate Conan binary preheat and restore for taos-gen (cherry-pick 3.3.6)
feat(ci): port all CI optimizations from main branch
…blic/tsdb!572)

ci: remove devcontainer configuration (3.3.6)
…488)

enh: make tdb page cache lock granularity smaller
enh: support specify vnode id when repair tdb
…-public/tsdb!598)

chore(cmake): default BUILD_CONTRIB and BUILD_USE_PUBLIC_DEPS to ON for standalone build (3.3.6)
fix(transport): fix memory leak in cliNotifyImplCb when sync response arrives...
…/tsdb!617)

fix: revert "make tdb page cache lock granularity smaller"
…lic/tsdb!591)

fix(packaging): handle taosws symlink for versioned and unversioned libs
…public/tsdb!621)

fix: narrow INSERT schema-retry condition to TK_NK_RP token

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces several key enhancements and bug fixes, including a new KILL COMPACT FORCE command to forcefully terminate compaction tasks, a backup mechanism for corrupted WAL files, Linux Release binary size optimizations, and a fix for tag name/value misalignment in TMQ metadata. It also enhances the vnode repair tool with secure temp file handling and adds comprehensive test coverage. The code review identified three important issues: a potential infinite loop or double-processing bug in hashset_remove due to in-place rehashing, a risk of path truncation in WAL file backups due to small buffer sizes, and ignored exit statuses for SSH commands in the repair tool.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment on lines +129 to +135
// Rehash subsequent entries to maintain probe chain integrity
for (size_t j = set->mask & (h + prime2); set->items[j] != 0; j = set->mask & (j + prime2)) {
size_t val = set->items[j];
set->items[j] = 0;
--set->nitems;
int nt = hashset_add_member(set, (void *)val);
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

In hashset_remove, rehashing subsequent entries in-place can lead to infinite loops or double-processing of the same item. If an item val from slot j is re-inserted at a slot k that is further down the probe chain (which was previously empty but is now filled by val), the loop will eventually reach k and process val again. To prevent this, all subsequent occupied slots in the cluster should be collected first, cleared from the hashset, and then re-inserted.

      // Collect subsequent entries in the same cluster to avoid infinite loops/double-processing
      size_t capacity = set->capacity;
      size_t *to_reinsert = tdbOsCalloc(capacity, sizeof(size_t));
      if (to_reinsert == NULL) {
        return 1;
      }
      int count = 0;
      for (size_t j = set->mask & (h + prime2); set->items[j] != 0; j = set->mask & (j + prime2)) {
        to_reinsert[count++] = set->items[j];
        set->items[j] = 0;
        --set->nitems;
      }
      for (int i = 0; i < count; i++) {
        int nt = hashset_add_member(set, (void *)to_reinsert[i]);
      }
      tdbOsFree(to_reinsert);

Comment on lines +389 to +390
char srcFile[WAL_FILE_LEN + TSDB_FILENAME_LEN];
char dstFile[WAL_FILE_LEN + TSDB_FILENAME_LEN];

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using WAL_FILE_LEN + TSDB_FILENAME_LEN for srcFile and dstFile buffers can lead to path truncation if the source or destination directory path is long (since tsWalCorruptionBackupDir can be up to PATH_MAX). It is safer to use PATH_MAX for both buffers to prevent buffer truncation and copy failures.

    char srcFile[PATH_MAX];
    char dstFile[PATH_MAX];

Comment on lines +196 to +221
FILE *fp = popen(cmd, "r");
if (fp == NULL) {
uError("repair: failed to run ssh command");
(void)close(fd);
(void)taosRemoveFile(localPathBuf);
return -1;
}
char buf[256];
while (taosGetsCmd(pCmd, sizeof(buf), buf) > 0) {

char buf[4096];
ssize_t nRead;
while ((nRead = fread(buf, 1, sizeof(buf), fp)) > 0) {
ssize_t nWritten = 0;
while (nWritten < nRead) {
ssize_t w = write(fd, buf + nWritten, nRead - nWritten);
if (w < 0) {
uError("repair: write to temp file failed: %s", strerror(errno));
(void)pclose(fp);
(void)close(fd);
(void)taosRemoveFile(localPathBuf);
return -1;
}
nWritten += w;
}
}
taosCloseCmd(&pCmd);
(void)pclose(fp);
(void)close(fd);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The exit status of the popen command is ignored. If the SSH command fails (e.g., connection timeout, authentication failure, or host offline), pclose(fp) will return a non-zero status, but the function currently proceeds as long as some data was read or the file is non-empty. Checking status != 0 ensures that any SSH command failure is correctly detected and handled. Additionally, fread returns size_t, so nRead should be declared as size_t to avoid signed/unsigned comparison warnings.

  FILE *fp = popen(cmd, "r");
  if (fp == NULL) {
    uError("repair: failed to run ssh command");
    (void)close(fd);
    (void)taosRemoveFile(localPathBuf);
    return -1;
  }

  char   buf[4096];
  size_t nRead;
  while ((nRead = fread(buf, 1, sizeof(buf), fp)) > 0) {
    ssize_t nWritten = 0;
    while (nWritten < (ssize_t)nRead) {
      ssize_t w = write(fd, buf + nWritten, (ssize_t)nRead - nWritten);
      if (w < 0) {
        uError("repair: write to temp file failed: %s", strerror(errno));
        (void)pclose(fp);
        (void)close(fd);
        (void)taosRemoveFile(localPathBuf);
        return -1;
      }
      nWritten += w;
    }
  }

  int status = pclose(fp);
  (void)close(fd);

  if (status != 0) {
    uError("repair: ssh command failed");
    (void)taosRemoveFile(localPathBuf);
    return -1;
  }

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants