Code sources

A code source is a selection of repositories, or whole groups, on one code host: GitLab (gitlab.com or a self-managed GitLab), GitHub (github.com) or Bitbucket Cloud. Olympusx AI reads it, removes secrets from every file, indexes the code into a vector database of the tenant, and maps it to one or more of the Code Assistant’s teams. A developer’s editor then searches the code of the team their key belongs to, and nothing else. Tenant admins add and manage sources on the Codebase knowledge tab.

Who can do what

Role	What they can do
Admin	Add, edit, sync and delete sources; turn on a source’s merge request webhook; set its review token.
Builder, Auditor	Read the sources, their repositories and how each last synced. Other roles see Only a tenant admin can add, change, sync or delete a source.
End user, Reviewer	Cannot open the page.

Before a source can be added

The tab lists what is missing under Before a source can be added. All four must hold:

• A codebase search model. A tenant admin chooses the embedding model under Codebase search on the Settings tab. Every source is indexed with it, so it cannot change while a source exists.
• The guardrails service. Code is never indexed without a scan for secrets. A platform admin installs and configures the guardrails service; see Configuration.
• The code parser. It cuts files along functions, methods, classes and types. It is installed with Olympusx AI and on by default; a platform admin who turned it off turns it back on in the chart values.
• A vector database that is ready. The source gets a collection of its own in it. You can create one from the wizard; a new vector database takes a few minutes to start. See Vector databases.

Add a source

1. Open Use cases › Code Assistant and select the Codebase knowledge tab.
2. Select Add source. The Add a codebase source wizard opens.
3. Provider. Choose GitLab, GitHub or Bitbucket. For GitLab, choose gitlab.com or Self-managed GitLab and enter its GitLab address with https, for example https://gitlab.example.test. One source reads from one provider.
4. Access. Enter an Access token (optional) and select Connect. The wizard shows Signed in as the token’s account and what it can list. Without a token, only public repositories can be indexed; GitHub and Bitbucket then answer 60 requests an hour, enough for a few small repositories.
5. Repositories. Choose what to index:
   • Select Show what the token can read, or enter an Owner and select Browse to list an owner’s repositories.
   • Choose a group’s All repositories in group, including new ones to take every repository in it, and any added later.
   • Or enter a path under Add by path and select Add.
6. Teams and storage.
   • Teams that search it: the Code Assistant’s teams whose developers can search this code.
   • Vector database: where the source’s index is kept. Create vector database makes one if the tenant has none.
   • Name (optional): left empty, the source is named after what it reads.
   • Sync: Daily at an hour of your choice in UTC (02:00 UTC by default), or Only when asked.
   • Advanced: Only these paths (optional) and Leave out these paths (optional) take comma-separated patterns, for example src/**,*.go.
7. Review. Check the summary, which tests the access and lists Repositories to index, then select Add source. The first sync starts at once.

The token each host needs

The token is kept sealed and never shown, logged or returned again.

Host	Token
GitLab	A group, project or personal access token with the read_api scope.
GitHub	A fine-grained token with Contents: Read and Metadata: Read, or a classic token with the repo scope for private repositories.
Bitbucket Cloud	A repository or workspace access token with repository read, or an Atlassian API token with your Account email. Leave the email empty for an access token.

How to name what you choose

Host	Group or owner	Repository path
GitLab	A group, a subgroup such as platform/backend, or a user	group/project, with any subgroups, or the project’s address
GitHub	An organization or a user, as in github.com/{owner}	owner/name, or the repository’s address
Bitbucket Cloud	A workspace, as in bitbucket.org/{workspace}	workspace/repository, or the repository’s address

What a sync does

1. Lists every file of the selected repositories on their default branch, within the path patterns.
2. Skips files that are too large (over 512 KiB), binary or empty.
3. Scans each new or changed file for secrets and replaces each one with ******. A file the scan could not check is left out of the index.
4. Cuts each file into chunks along its functions, methods, classes and types, embeds them with the codebase search model, and records what each file imports, calls and inherits for the repository graph.
5. Removes from the index the files the source no longer lists.

Code is scanned for secrets before it is indexed, and ordinary code is indexed exactly as written, so every symbol and string in search results matches the source.

Languages

Files in these languages are cut along their structure, and their symbols appear in search and in the graph: Bash, C, C++, C#, Dockerfile, Go, HCL, Java, JavaScript (with JSX), JSON, Kotlin, Markdown, PHP, Python, Ruby, Rust, Scala, SQL, Swift, Terraform, TOML, TypeScript (with TSX) and YAML.

Relations between code (imports, calls and inheritance) are read for C, C++, C#, Go, Java, JavaScript, Kotlin, PHP, Python, Ruby, Rust, Scala, Swift and TypeScript.

Other text files are indexed in windows of lines, without symbols, and are searchable. The repository page counts them as Other (not parsed).

Read a source

The Sources list shows each source’s Source, Provider, What it reads, Repositories, Teams, Last sync, Files, Chunks and Next sync. Expand a row to see:

• Files indexed, Skipped in the last sync, Removed in the last sync and Files with secrets removed.
• Why the last sync failed, when it did.
• The Embedding model, the Only and Left out patterns, and when it was Added.
• The Merge request webhook and Review token state. See Merge request review.
• The source’s repositories, each linking to its repository page.

Change, sync or delete a source

From a source’s actions menu:

• Sync now starts a sync. Only one sync of a source runs at a time; a second request says Already syncing.
• Edit changes the repositories it holds, its token, the teams that search it and when it syncs. Its provider, vector database and name stay. Under Token, choose Keep the current token, Replace it with a new token or Remove it: read public repositories only. The index follows a change at the next sync; turn on Sync after saving to see it sooner.
• Delete removes the source’s index and its collection; no editor finds this code any more. The repositories on the host are not touched. Adding them again indexes them from the start.

Check it worked

• Last sync shows a time and no failure, and Files is above zero.
• Open a repository and search it on Graph and search, or ask a question in a developer’s editor with the codebase tools configured.

When it does not work

Symptom	Cause and fix
Add source is unavailable	A prerequisite is missing; the tab names it.
Provider did not accept the connection	The token is wrong, expired or lacks the scope above.
Nothing by that name can be read with this access	The owner does not exist, or the token cannot see it.
A source takes at most 100 repositories and groups.	Choose a group to take all of its repositories at once, or add a second source.
The sync fails with more than 50,000 indexable files	Narrow the source with Only these paths or Leave out these paths, or name smaller repositories.
The sync stops after 30 minutes	A first sync of a large group downloads, scans and embeds every file. Split the source, or narrow it with path patterns.
Files with secrets removed is above zero	Expected: those files were indexed with their secrets replaced. Rotate any real credential the scan found in your repositories.

Limits

Limit	Value
Groups and repositories one source names	100
Indexable files in one source	50,000
Largest file indexed	512 KiB
One sync	30 minutes
Requests to GitHub or Bitbucket without a token	60 an hour (the host’s limit)
Daily sync hour	0 to 23, UTC
Source name	One line, at most 100 characters

Related

• The repository view
• Merge request review
• Security code assessments, which read the code a source indexes
• Vector databases