首頁 » Linux » seafile

1. seafile usage

由 datahunter 在 二, 28/02/2017 - 12:38 發表

最後更新: 2018-02-28

 

介紹

HomePage: https://www.seafile.com/

中文 BBS: https://bbs.seafile.com/

Forum: https://forum.seafile.com/

目錄

 

注意事項

 * Restore Delete 了的 file, 它的 History 會唔見晒
 * 當出現與 Delete 了的 file 同名檔案時, 那 Trash 內的那個會不見了

 

Components

 

Seahub (django): the web frontend.

By default, Seahub runs as an application within gunicorn

 * All access to the Seafile service (including Seahub and Seafile server) can be configured behind Nginx or Apache

    (This way all network traffic to the service can be encrypted with HTTPS)

 * When user accesses Seahub, Seahub accesses data from Seafile server using Ccnet RPC

Seafile server (seaf-server): data service daemon

                                            - C programming

                                            - handles raw file upload, download and synchronization (port 8082)

Ccnet server (ccnet-server): RPC service daemon. It is only used for internal communication. (client & server side)

                                            enable communication among multiple components.

seafile-controller: monitors ccnet and Seafile daemons, restarts them if necessary

 

Data Model

 

Repo

A repo is also called a library. Every repo has an unique id (UUID), and attributes like description, creator, password.

Branch

In PC client, modifications will first be committed to the local branch.

Then the master branch is downloaded from server, and merged into local branch.

After that the local branch will be uploaded to server.

Then the server will fast-forward its master branch to the head commit of the just uploaded branch.

When users update a repo on the web,

modifications will first be committed to temporary branch on the server, then merged into the master branch.

Commit

Like in GIT.

FS

There are two types of FS objects, SeafDir Object and Seafile Object

Block

A file is further divided into blocks with variable lengths.

 * deduplicate data between different versions of frequently updated files
 * transferring data to/from multiple servers in parallel.

indexing

After a file is uploaded, it needs to be divided into fixed size blocks and stored into storage backend.

By default, the file server uses 1 thread to sequentially index the file and store the blocks one by one.

 

Synchronization algorithm

 

Each downloaded repo is bound to an ordinary local folder("worktree").

synchronization work flow

1. Seafile client daemon detects changes in the worktree (via inotify etc).
2. The daemon commits the changes to the local branch.
3. Download new changes from the master branch on the server (if any).
4. Merge the downloaded branch into local branch (also checkout changes to worktree).
4. Fast-forward upload local branch to server's master branch.

Interrupted synchronization

Seafile 會用 'Git's index file' 去 check worktree

'Git's index file' - caches the timestamps of every file in the worktree when the last commit is generated.

Merge 的處理

"redoing" the merge carefully after restart.

starting merge until no file is write-protected in the worktree.

 

Desktop Syncing Clients

 

  • Desktop Syncing Clients
  • Drive Client

Drive Client

By default, "S:" will be used.

The file contents are not downloaded at this moment.
So it'll not take very long. You can let it run in the background.
You'll be noticed when the fetch is done.

File contents will be downloaded when you open a file.
Recently opened file will be cached in your local disk.

All cached files will be marked with a green tick.

The Drive client regularly cleans up unused cached files in the background.
You can also limit the local cache space.

Why can't I create new files or delete folders in root folder

The root folder of the virtual drive contains only library folders. Seafile can only sync files inside libraries.

Files in the root folder cannot be uploaded to Seafile server. So it doesn't make sense to support creating files in root folder.

Files deleted

In Seafile Web App, navigate into the parent folder containing the deleted file or folder.

Click the "trash" icon in the library operation bar. The deleted files and folders will be listed in deletion time order.

In the bottom of the list, you can click the "more" button to load more deleted files.

In the deleted files list, you can restore file and folders. If a file or folder's parent folder has been deleted,

it will be restored to the library's root folder.

Note that the trash bin only contains files and sub-folders deleted under the current folder.

Deleted Library

Unlike the files and folders in the library, a deleted library currently cannot be restored by the user.

But the Seafile server's administrator can restore a deleted library for users.

After a library is deleted, it has following effects:

     * Share of the files and folders within this library are all canceled.
     * Desktop clients synced with this library cannot sync with it any more.
        But the local files on the client won't be deleted.

 

Client

Syncing Clients

seaf-daemon.exe:        Ram: 11 Mb

seafile-applet.exe:       Ram: 58 Mb

Drive Clients

seadrive.exe: RAM: 7 Mb

seadrive-gui.exe Ram: 53 Mb

Mobile Clients

https://play.google.com/store/apps/details?id=com.seafile.seadroid2

https://itunes.apple.com/cn/app/seafile-pro/id639202512?l=en&mt=8

 

Sync Library To Local Folder

# 當 sync 名叫 "test" 的 library 到 G:/Seafile 時,

如果在 G:/Seafile 內已有 Folder "test"

那 contents will be merged

 - 本地刪除了的檔案會再下載一次回來

 - 本地同名檔案會在檔尾加上 (SFConflict [email protected] 2018-11-19-12-47-33),

即使它比 remote 的還要新, 及選了 "sync with an existing folder"

 

Encryption

 

File contents in encrypted libraries are encrypted on client side. (metadata which are currently not encrypted)

The encryption password is not stored on the server.

When creating an encrypted library:

create an encrypted library in the web app,

the password is sent to the server.

The server use this password to create the library.

But it doesn't store the plain text password.(cache the password in memory for one hour.)

create an encrypted library with the desktop client

the password is not sent to the server.

過程:

1. Generate a 32-byte long cryptographically strong random number.("file key")
2. Encrypt the file key with the user provided password.
3. All file data is encrypted by the "file key" with AES 256/CBC

 

Security

 

File URL

When a file download link is clicked, a random URL is generated for user to access the file from fileserver.

This url can only be access once. After that, all access will be denied to the url.

User 's login password

PBKDF2SHA256$iterations$salt$hash

4 parts by the $ sign

 

File permission management

 

Seafile manages files using libraries. Every library has an owner,

who can share the library to other users or share it with groups.

The sharing can be read-only or read-write.

Read-only libraries can be synced to local desktop.

The modifications at the client will not be synced back.

If a user has modified some file contents, he can use "resync" to revert the modifications.

 

Seafile FSCK

 

On the server side, Seafile stores the files in the libraries in an internal format.

Seafile has its own representation of directories and files (similar to Git).

"head commit" = current state of the library

# Checking Integrity

./seaf-fsck.sh

# check integrity for specific libraries

./seaf-fsck.sh [library-id1] [library-id2] ...

# Repairing Corruption

./seaf-fsck.sh --repair [library-id1] [library-id2] ...

* If the library state (commit) recorded in database is not found in data directory,
   find the last available state from data directory.

* Check data integrity in that specific state. If files or directories are corrupted,
   set them to empty files or empty directories.
   The corrupted paths will be reported,
   so that the user can recover them from somewhere else.

# Example

./seaf-fsck.sh > fsck.txt

# Exporting Libraries to File System

./seaf-fsck.sh --export top_export_path [library-id1] [library-id2] ...

會一個個 Libraries 放在 export_path

i.e.

mkdir /mnt/seafile_export

chown seafile. /mnt/seafile_export

sudo -u seafile ./seaf-fsck.sh --export /mnt/seafile_export

 

seaf-fuse

 

在服務器查看用戶上傳的文件

seaf-fsck
seaf-fuse

seaf-fuse

Encrypted folders can't be accessed by seaf-fuse.
'read-only'

Step

mkdir /mnt/seafile-fuse
cd /home/seafile/seafile-server-latest
./seaf-fuse.sh start /mnt/seafile-fuse
./seaf-fuse.sh stop

Layout

tree /mnt/seafile-fuse

/mnt/seafile-fuse
└── [email protected]
    └── 5b9d4dc1-ca53-4cde-8ce4-d3ffbdf0a385_MyFile
        ├── test1
        │   ├── MPIO.png
        │   └── test3
        │       └── test.txt
        └── test2
            └── 1.pdf

5 directories, 3 files

 

fail2ban

 

yum install -y fail2ban

Each time a user/computer tries to connect and fails 3 times,
a new line will be write in your seafile logs (seahub.log)

maxretry (default to 3 is equivalent to 5 real attemps in seafile ) 因為第 3 次時才有 log

Log

# V9

... [WARNING] seahub.auth.views:153 login Login attempt limit reached, show Captcha, email/username: test, ip: 192.168.123.1, attemps: 5

/etc/fail2ban/jail.local

[seafile]
enabled  = true
port     = http,https
filter   = seafile-auth
logpath  = /home/seafile/logs/seahub.log
maxretry = 3
findtime  = 300
bantime = 3600

/etc/fail2ban/filter.d/seafile-auth.conf

[INCLUDES]
before = common.conf
[Definition]
_daemon = seaf-server
failregex = Login attempt limit reached, .*, ip: <HOST>, .*
ignoreregex =

Test the filter

fail2ban-regex /home/seafile/logs/seahub.log /etc/fail2ban/filter.d/seafile-auth.conf

systemctl enable fail2ban --now

Testing

 

fail2ban-client status seafile

Status for the jail: seafile
|- Filter
|  |- Currently failed: 1
|  |- Total failed:     4
|  `- File list:        /home/seafile/logs/seahub.log
`- Actions
   |- Currently banned: 1
   |- Total banned:     1
   `- Banned IP list:   192.168.123.200

fail2ban-client set seafile unbanip 192.168.123.200

 

Garbage Collection

 

To release the storage space occupied by unused blocks

 * Before running GC, you must shutdown the Seafile program on your server
    (new blocks written into Seafile while GC is running may be mistakenly deleted by the GC program)

Notes

libraries deleted by the users are not immediately removed from the system.
Instead, they're moved into a "trash" in the system admin page.

Before they're cleared from the trash, their blocks won't be garbage collected.

會被 cleanup 的 Blocks

 - Blocks that no library references to, that is, the blocks belong to deleted libraries

 - Out-dated blocks (history length limit) [ "-r" option ]

Usage

seaf-gc.sh [--dry-run] [repo-id1] [repo-id2] ...

e.g.

seaf-gc.sh --dry-run

# remove libraries garbage blocks

seaf-gc.sh

# deleted libraries & libraries for outdated historic blocks

seaf-gc.sh -r

/etc/cron.weekly/seafile

#!/bin/bash

PathToSeafile=/home/seafile/seafile-server-latest

# stop the server
echo "Stopping the Seafile-Server..."
systemctl stop seahub seafile

echo "Giving the server some time to shut down properly..." && sleep 10

# run the cleanup
echo Seafile cleanup started...
sudo -u seafile $PathToSeafile/seaf-gc.sh -r

echo "Giving the server some time..." && sleep 3

# start the server again
echo "Starting the Seafile-Server..."
systemctl start seafile seahub

echo "Seafile cleanup done!"

chmod 700 /etc/cron.weekly/seafile

systemctl restart crond             # Debian: systemctl restart cron

 

Databases

 

DB

ccnet-db: contains user and group information
seafile-db: contains library metadata
seahub.db: contains tables used by the web front end (seahub)

User Table

ccnet-db

- EmailUser (is_staff = admin)

seafile-db

 - UserQuota (當 user 不是用 default Quota 時)
 - RepoOwner    
 - Repo (有什麼 "Libraries") [https://domain/#my-libs/lib/1e0db1ef-7e8a-x-x]

    seafile-data/storage/fs/repo_id
    seafile-data/storage/blocks/repo_id
    seafile-data/storage/commits/repo_id

seahub-db

 - api2_tokenv2 (connect 了的 device)
 - base_userstarredfile (加了星的 file)
 - share_fileshare (expire_data)
 - share_uploadlinkshare

 

Backup

 

Seafile Library data

Path: /home/seafile

最安全的 backup 方案

Steps

1. Stop Seafile service first

2. Backup the DB & Data;

3. Start Seafile service again

 * 不 stop service 時的 backup 方案

backup SQL first, data directory later

Since you backup database first, all records in the database have valid objects to be referenced.

So the libraries won't be corrupted.

But new objects written to storage when you're backing up are not referenced by database records.

So some libraries are out of date. When you restore, some new data are lost.

seafile-backup.sh

#!/bin/bash
# create by tim@20170410

#### Setting ####
_INSTALLPATH=/home/seafile
_BAKROOT=/backup
_dbuser=seafile
_dbpass=?

#### Code ####
_NOW=`date +"%Y-%m-%d_%H-%M-%S"`
_BAKDBTO=$_BAKROOT/bak_$_NOW/db
_BAKDATATO=$_BAKROOT/bak_$_NOW/data

mkdir -p $_BAKDBTO $_BAKDATATO

# DB
echo "Start Backup DB ..."
mysqldump -u$_dbuser -p$_dbpass --skip-add-drop-table ccnet-db > $_BAKDBTO/ccnet-db.sql
mysqldump -u$_dbuser -p$_dbpass --skip-add-drop-table seahub-db > $_BAKDBTO/seahub-db.sql
mysqldump -u$_dbuser -p$_dbpass --skip-add-drop-table seafile-db > $_BAKDBTO/seafile-db.sql

cd $_BAKDBTO
gzip ccnet-db.sql seafile-db.sql seahub-db.sql &

# File
echo "Start Backup File ..."
cp -aLl $_BAKROOT/Last/data/seafile $_BAKDATATO 2> /dev/null
rsync --delete -a $_INSTALLPATH $_BAKDATATO

# link to last
rm $_BAKROOT/Last 2> /dev/null
ln -s $_BAKROOT/bak_$_NOW $_BAKROOT/Last

# backup result
echo $_NOW > $_BAKROOT/log.txt
du -sh $_BAKROOT/Last/data >> $_BAKROOT/log.txt
ls -lh $_BAKROOT/Last/db/*sql.gz | awk '{print $5,$9}' >> $_BAKROOT/log.txt
cat $_BAKROOT/log.txt

# END
echo "Done"

Configure backup

 * seahub_settings.py 內有 "SECRET_KEY"

 * ccnet.conf 內有 "ID"

 

Data Folder Structure

 

 

seafile-data/
├── commits
├── fs
├── httptemp
├── library-template
│   └── seafile-tutorial.doc
├── storage
│   ├── blocks
│   ├── commits
│   └── fs
└── tmpfiles

 

Forgot Admin Account

 

# Forgot Admin Account

reset-admin.sh

 

Cleanup

 

# Clear expired session (records in Seahub database)

./seahub.sh clearsessions

# Cleanup DB

# V6.2.5

use seahub_db;
DELETE FROM notifications_usernotification WHERE to_days(now()) - to_days(timestamp) > 90;
DELETE FROM api2_tokenv2 WHERE to_days(now()) - to_days(last_accessed) > 90;

 

Webdav

 

 * By default, seafdav is disabled.
 * The access to the webdav server may be slow sometimes.
   (more suitable for infrequent file access)

Sample Configuration 1: No nginx/apache

seafdav.conf

[WEBDAV]
enabled = true
port = 8080
fastcgi = false
share_name = /

 

SMTP Setting

 

# Options of Email Sending

seahub_settings.py

EMAIL_USE_TLS = False
EMAIL_HOST = 'smtp.example.com'             # smpt server
EMAIL_PORT = 25

EMAIL_HOST_USER = '[email protected]'    # username and domain
EMAIL_HOST_PASSWORD = 'password'            # password

DEFAULT_FROM_EMAIL = EMAIL_HOST_USER
SERVER_EMAIL = EMAIL_HOST_USER

Note:

If your email service still does not work, you can checkout the log file logs/seahub.log to see what may cause the problem.

If you want to use the email service without authentication leaf EMAIL_HOST_USER and EMAIL_HOST_PASSWORD blank ('')

REPLACE_FROM_EMAIL

# This only affects email sending for file share link.

# Replace default from email with user's email or not, defaults to ``False``

REPLACE_FROM_EMAIL = True

ADD_REPLY_TO_HEADER

# Set reply-to header to user's email or not, defaults to ``False``. For details,

# please refer to http://www.w3.org/Protocols/rfc822/

ADD_REPLY_TO_HEADER = True

 

收費版

 

3 User => Free

價格:

https://www.seafile.com/en/product/private_server/

 

Troubleshoot

 

Q1

[03/01/17 16:49:09] http-tx-mgr.c(748): libcurl failed to GET

https://album.datahunter.org:8443/seafhttp/protocol-version: SSL peer certificate or SSH remote key was not OK.

A1

SSL 的 CN 不對應 Client 連的 URL

------

Q2

load 不到 css

http://xxx:8000/media/CACHE/css/ac5eb489f018.css

A2

clear cache

stop seafile server
delete the folder "/tmp/seahub_cache"
start seafile server
clear browser cache and cookies
login again

 

WSGI mode

 

# From 6.2, It is recommended to use WSGI mode for communication between Seahub and Nginx

location / {
         proxy_pass          http://127.0.0.1:8000;
         proxy_set_header    Host $host;
         proxy_set_header    X-Real-IP $remote_addr;
         proxy_set_header    X-Forwarded-For $proxy_add_x_forwarded_for;
         proxy_set_header    X-Forwarded-Host $server_name;
         proxy_read_timeout  1200s;

         # used for view/edit office file via Office Online Server
         client_max_body_size 0;

         access_log      /var/log/nginx/seahub.access.log;
         error_log       /var/log/nginx/seahub.error.log;
    }

 

DB & Table

 

ccnet-db

EmailUser
Group
GroupUser
UserRole
...

seahub-db

api2_tokenv2
django_session
share_fileshare
share_uploadlinkshare
two_factor_X
...

seafile-db

RepoInfo
RepoOwner
UserQuota
RepoUserToken
...

 

Path for download desktop client

 

https://datahunter.org/nginx_configure#HttpAutoindexModule

 

Windows Client Config

 

seafile

  • HKCU\SOFTWARE\Seafile\Seafile Client

seadrive

  • HKCU\SOFTWARE\SeaDrive\Seafile Drive Client\Settings\
  • C:\Users\USERNAME\seadrive

 

Doc

 

 

 

»
  • 瀏覽次數： 5925

Creative Commons license icon Creative Commons license icon