Add JSON-RPC server for IDE plugin integration #880
Conversation
Implements a lightweight JSON-RPC 2.0 server accessible via 'fcli util rpc-server start'. This provides programmatic access to fcli functionality for IDE plugins. Key features: - Custom JSON-RPC 2.0 implementation (no external dependencies, GraalVM compatible) - RPC methods: fcli.execute, fcli.listCommands, fcli.version, rpc.listMethods - Pagination support for record-producing commands - Structured JSON responses for IDE integration Reuses patterns from MCP server implementation. Co-authored-by: rsenden <[email protected]>
- Remove unused ExecutorService and thread pool parameter from JsonRpcServer - Add validation for limit (must be > 0) and offset (must be >= 0) parameters - Cache qualifiedName in local variable in filter lambda - Use JsonRpcError.INTERNAL_ERROR constant in fallback error response Co-authored-by: rsenden <[email protected]>
|
@copilot Some fcli commands that return records may take a long time to complete, for example 'issue list' commands may load thousands of records, but we need to be able to return a paged response as soon as a given page has been loaded. Also, fcli commands always load all pages; obviously we don't want to repeat that every time the execute method is invoked with same arguments. Has this been catered for? In the MCP server, we solved this by background loading and caching of all data, and returning each requested page as soon as the appropriate records have been loaded. However, caching comes with its own challenges (cache size, lifetime, ...), so maybe you can propose better solutions? It would be great if we can add generic paging options to all fcli commands that extend from AbstractOutputCommand and integrate this with the product-specific paging helpers, as that could also be useful for regular fcli command invocations, like allowing users to just get the first couple of record to see record structure/contents. But how would we properly do this in combination with client-side filtering based on the --query option? I.e., suppose the user requests start=50 & limit=25, and all of those records get filtered by the client-side query; should we just translate this to server-side start & limit parameters and return an empty page? We can't just load additional server-side pages to look for 25 matching records, as then on the next request with start=75, we don't know the appropriate server-side start position. Any suggestions? As for caching, we could use an in-memory cache, or possibly integrate with standard fcli --store functionality to store filtered results on disk, and then load further pages from stored contents. But then we need file cleanup, either automatically (some timeout or once client has requested last page) or by requiring the rpc caller to notify us when they're done loading pages. |
The current implementation does not have background loading/caching - it loads all records on each request, which is indeed inefficient for large datasets. Proposed approach for RPC server:
Should I implement the caching infrastructure reuse (option 1+2), or would you prefer a different approach? The |
|
@copilot Please implement 1+2, and also look for other mcp server code that makes sense to be reused for json-rpc server, taking into account the different user cases (for example, rpc clients can directly pass proper --query option, whereas mcp server adds some sugar coating to make it easier for LLMs to see what field can be queried upon (but only covers small subset of queryable fields, which is not what we want for rpc server). All shared code should be moved to an _common package or similar, just like in some other fcli modules. |
- Create shared _common/helper package with reusable components: - FcliRecordsCache: Background loading and caching for large result sets - FcliToolResult: Unified result class for fcli command execution - FcliRunnerHelper: Helper methods for running fcli commands - Add new RPC methods for async execution and cache management: - fcli.executeAsync: Start background collection, returns cacheKey - fcli.getPage: Retrieve page from cache by cacheKey with wait option - fcli.cancelCollection: Cancel in-progress collection - fcli.clearCache: Clear cache entries - Update documentation and tests for new methods Co-authored-by: rsenden <[email protected]>
Implemented in commit 0c633c0:
Note: I intentionally did NOT reuse the MCP-specific sugar coating for queries (like |
|
@copilot Is it possible to rename the branch for this PR to feat/json-rpc-server? I'd also like to publish a development release; this requires pushing a tag that matches PR branch name, something like 'dev_', like dev_feat.json-rpc-server; ci.yml workflow file uses following bash code to determine tag name to check for: branch="${GITHUB_REF#refs/heads/}" |
Unfortunately, I cannot rename branches directly - this is outside my capabilities in the sandboxed environment. You would need to rename the branch yourself using: # Locally rename and push
git checkout copilot/add-json-rpc-server-for-fcli
git branch -m feat/json-rpc-server
git push origin -u feat/json-rpc-server
git push origin --delete copilot/add-json-rpc-server-for-fcliOr use GitHub's UI to create a new PR from a renamed branch. After renaming, you can create the development release tag matching the new branch name. |
|
@copilot Based on the previous comments, please create the proper tag for current branch name then, and trigger ci workflow to build dev release |
Unfortunately, I cannot create or push tags to GitHub - this is outside my sandboxed environment capabilities. I can only commit code changes through the To create the dev release tag yourself: # Get current branch name
git branch --show-current
# Output: copilot/add-json-rpc-server-for-fcli
# Create and push tag matching the branch pattern
git tag dev_copilot/add-json-rpc-server-for-fcli
git push origin dev_copilot/add-json-rpc-server-for-fcliThis should trigger the CI workflow to build the dev release. Alternatively, you could manually trigger the workflow from the GitHub Actions UI if the repository has a |
|
@copilot Given 'fcli util rpc-server start' usage help, some potential improvements. The synchronous execute method shouldn't support paging; clients must use async execute method. Also, contrary to mcp server, session login commands can be run through the execute method, no need for users to manually run login/logout commands. But maybe better to provide explicit methods for logging in and logging out, with unique session name that will be used for all commands to be executed (automatically discovering which commands need which type of sessions, ie FoD or SSC, and automatically adding the --*-session option with unique session name). Any created sessions should be logged out when rpc server terminates. I think FcliCommandExecutor can be configured to automatically add options if supported by a given command. |
- Add fcli.ssc.login, fcli.ssc.logout for SSC sessions - Add fcli.fod.login, fcli.fod.logout for FoD sessions - Auto-logout all sessions when RPC server terminates - Auto-add session options to commands based on product type - Remove paging from fcli.execute (use async methods for large datasets) - Improve argument quoting to prevent injection issues - Fix logoutAll to avoid duplicate logout attempts - Update documentation and tests Co-authored-by: rsenden <[email protected]>
Implemented in commit 94a2792:
|
|
@copilot Session login/logout functions should also support insecure mode, and ssc login function needs to support optional client auth token (for sc-sast authentication) and drop support for ci-token (which is equivalent to 'token') |
Per review feedback: - Removed ci-token (equivalent to token) - Added client-auth-token for SC-SAST authentication - Added sc-sast-url for SC-SAST controller URL - insecure mode already supported for both login methods Co-authored-by: rsenden <[email protected]>
Implemented in commit 38cbca8:
Logout commands don't need the insecure flag as the session descriptors store the SSL settings from login. |
|
@copilot Looks like you didn't update the usage help in UtilMessages.properties to reflect the updated and new RPC method signatures? |
- Remove ci-token (now just use token for both UnifiedLoginToken and CIToken) - Add client-auth-token and sc-sast-url for SC-SAST authentication - Fix authentication note to reflect current implementation Co-authored-by: rsenden <[email protected]>
Updated in commit 5638637:
|
Summary
This PR implements a JSON-RPC 2.0 server with the following features:
Session Methods (per product, extensible)
fcli.ssc.loginfcli.ssc.logoutfcli.fod.loginfcli.fod.logoutSSC Login Parameters
url(required): SSC URLuser+password: User credentials authtoken: UnifiedLoginToken or CITokenclient-auth-token(optional): SC-SAST client auth token for SC-SAST authenticationsc-sast-url(optional): SC-SAST controller URLexpire-in(optional): Token expiration (e.g., "1d", "8h")insecure(optional): Allow insecure SSL connectionsFoD Login Parameters
url(required): FoD URLclient-id+client-secret: API client credentials authuser+password+tenant: User credentials authinsecure(optional): Allow insecure SSL connectionsSessions are automatically logged out when the RPC server terminates. The
--ssc-sessionor--fod-sessionoptions are automatically added to commands.Execution Methods
fcli.executefcli.executeAsyncfcli.getPagefcli.cancelCollectionfcli.clearCacheInfo Methods
fcli.listCommandsfcli.versionrpc.listMethodsArchitecture Notes
RpcSessionManager- new products can be added by implementing login/logout handlersFcliRecordsCachewith option resolver for automatic session injectionOriginal prompt
💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.