Removing unwanted tarballs from the dist-git rpms

Description

This SOP provides instructions for sysadmin-main level users to remove unwanted tarballs and source RPMs from a branch in the dist-git repositories on Fedora’s infrastructure.

Access level required

sysadmin-main

Machine

pkgs01.rdu3.fedoraproject.org

Steps to access the machine

SSH into batcave01.rdu3.fedoraproject.org:
```
   ssh batcave01.rdu3.fedoraproject.org
```
Switch to the root user:
```
   sudo su
```
SSH into pkgs01.rdu3.fedoraproject.org:
```
   ssh pkgs01.rdu3.fedoraproject.org
```

Preparing non-bare repository before removing.

Before completely removing the file from dist-git history we need to add it to the lookaside cache. And ensure that the file is presented in .gitignore and sources files.

Adding file into lookaside cache

Clone the package repository:
```
   fedpkg clone PKG_NAME
```
Enter the cloned directory:
```
   cd PKG_NAME
```
Add the file into lookaside cache:
```
   fedpkg upload FILE_NAME
```

Adding file into .gitignore if it’s not already present

Add the file name to .gitignore:

   echo FILE_NAME >> .gitignore

Adding SHA512 hash into sources file if it’s not already present

Generate SHA512 hash:
```
   sha512sum FILE_NAME >> sources
```

Copy the hash into sources with specific format:

   echo "SHA512 ($FILE_NAME) = $SHA512_HASH" >> sources

Commit and push all changes above

Removing unwanted tarballs from dist-git

Option 1: Manual

To remove unwanted tarballs and source RPMs from a specific branch of a package in the dist-git repository:

Navigate to the package repository:

   cd /srv/git/repositories/rpms/PKG_NAME.git

Disable warning messages:
```
   FILTER_BRANCH_SQUELCH_WARNING=1
```

Rewrite history on the desired branch:

   git filter-branch --force --index-filter 'git rm -r --cached --ignore-unmatch *.src.rpm *.tar.gz' --original refs/archive -- BRANCH

Replace *.src.rpm *.tar.gz with the actual file name, that you would like to remove. If you want to remove file from all branches use --all instead of BRANCH.

Move the backup reference:

   mv refs/archive/refs/heads/BRANCH refs/archive/BRANCH

Remove the redundant directories:
```
   rm -r refs/archive/refs/
```

Replace PKG_NAME with the name of the package. Replace BRANCH with the target branch name.

Verification

Verify the unwanted files have been removed from the branch history.

Run Verification

    git for-each-ref --format='%(refname:short)' | while read branch; do
        echo "Checking $branch:"
        git ls-tree -r $branch --name-only | grep -q "$FILE_NAME" && echo "FOUND in $branch" || echo "Not in $branch"
    done

Use actual file name instead of $FILE_NAME.

Option 2: Automated

We will use a script from releng repository to remove unwanted tarballs from dist-git.

Navigate to the root directory for package repositories:
```
   cd /srv/git/repositories/rpms/PKG_NAME.git
```
Create tmp directory and cd into it:
```
   mkdir tmp
   cd tmp
```

Get the script from releng repository:

   curl -O https://forge.fedoraproject.org/releng/tooling/src/branch/main/packages/maintenance/removing_unwanted_tarball.sh

Make the script executable:

   chmod +x removing_unwanted_tarball.sh

Replace REPO_NAME with the name of the package. It should start with ../ if you are running the scripts from tmp directory.
Replace FILE_NAME with the name of the file you want to remove.
Replace USER_OWNER with the user owner for directories inside the package repository.
It should be done because the scripts could be run by different users and we want ownership to be consistent.
Replace GROUP_OWNER with the group owner for directories inside the package repository.

Run the script:
```
   ./removing_unwanted_tarball.sh
```

Want to help? Learn how to contribute to Fedora Docs ›