Merge branch 'server_build' into main

ComPlat · Jun 14, 2023 · f0eeb5a · f0eeb5a
2 parents 2070866 + 359ea8d
commit f0eeb5a
Show file tree

Hide file tree

Showing 24 changed files with 459 additions and 183 deletions.
diff --git a/.gitignore b/.gitignore
@@ -10,3 +10,4 @@
 /193.196.55.54.crt
 /server.crt
 /temp/
+/bin/
diff --git a/README.md b/README.md
@@ -4,16 +4,18 @@
 Once all files in a subdirectory <CMD arg -src> 
 (or a file directly in <CMD arg -src>) have not been
 modified for about exactly <CMD arg -duration> seconds,
-the subdirectory is sent to a remote WebDAV server at <CMD arg -dst>.
+the subdirectory is sent to a remote WebDAV or SFTP server at <CMD arg -dst>.
 
-**Important** this project has to be compiled with go version 1.10.8. Otherwise, it cannot be guaranteed to run on Win XP.
+**Important** 
+1) If you want to run this tool on Windows XP it has to be compiled with go version 1.10.8.
+2) If the tool is used with the SFTP protocol under Windows XP, you must also save a portable version of If the tool is used with the SFTP protocol under Windows XP, you must also save a protable version of WinSCP (WinSCP)[https://winscp.net/download/WinSCP-5.21.5-Portable.zip] in the same folder as the efw.exe. in the same folder as the efw.exe.
 
 ## Usage
 
 efw -duration &lt;integer&gt; -src &lt;folder&gt; -dst &lt;url&gt;/ -user &lt;username&gt; -pass &lt;password&gt; [-zip]
 
-    -crt [string]
-        Path to server TLS certificate. Only needed if the server has a self signed certificate.
+    -name string
+        Name of the EFW instance. This name is used to create a working folder in the user home directory. .
 
     -duration [int]
         Duration in seconds, i.e., how long a file must
@@ -33,51 +35,30 @@ efw -duration &lt;integer&gt; -src &lt;folder&gt; -dst &lt;url&gt;/ -user &lt;us
         WebDAV Password
 
     -user [string]
-        WebDAV user
+        WebDAV user or SFTP user
 
     -type [string: file|folder|zip]
          Type must be 'file', 'folder' or 'zip'. The 'file' option means that each file is 
          handled individually, the 'folder' option means that entire folders are transmitted
          only when all files in them are ready. The option 'zip' sends a folder zipped, only
          when all files in a folder are ready.
+
+    -crt (Optional) [string]
+         Path to server TLS certificate. Only needed if the server has a self signed certificate.
 
 ## Setup the efw on a Windows system
-1) Download the **run_example.bat**, the **efw.exe** and the **task_example.vbs** for your system [here](https://github.com/ComPlat/ELN_file_watcher/releases/tag/latest)
-   - In the following introduction we will assume a 64 bit Windows system.
-2) Copy the **efw_win64.exe** and the **run64_example.bat** to the target directory on your target machine
+1) Download the **efw_run_example.bat**, the **efw.exe** and the **task_example.vbs** for your system [here](https://github.com/ComPlat/ELN_file_watcher/releases/tag/latest)
+2) Copy the **efw_{system}.exe** and save it **as efw.exe**. Additionally, download the **efw_run_example.bat** to the target directory on your target machine
    - In the following we use the example "C:\Program Files\file_exporter".
 3) Replace in the **task_example.vbs**:
-   - "&lt;Full path to run_.bat&gt;" with "C:\Program Files\file_exporter\run64_example.bat"
-4) Replace in the **run64_example.bat**:
-   - &lt;Path to efw_win64.exe&gt; with "C:\Program Files\file_exporter\"
-   - Setup all parameter (hint: use _efw_win64.exe -h_):
-   - -dst, -src, -crt, -duration, -user, -pass, -crt, -zip
+   - "&lt;Full path to run_.bat&gt;" with "C:\Program Files\file_exporter\efw_run_example.bat"
+4) Replace in the **efw_run_example.bat**:
+   - &lt;Path to efw.exe&gt; with "C:\Program Files\file_exporter\"
+   - Setup all parameter (hint: use _efw.exe -h_):
+   - -dst, -src, -crt, -duration, -user, -pass, -crt, -zip, -name, -transfer, -type
 5) copy the **task_example.vbs** into the startup directory
    - Hint: **Windows Key + R** to open run and type **shell:startup**. This will open Task Scheduler
 
-In case it is not working it might be that either the log file or the source directory can not be accessed by the executing user. In this case please make sure the log file and the src dirctory have the correct read & writing permissions.
-A tutorial how to set permissions can be found at: [Microsoft answers](https://answers.microsoft.com/en-us/windows/forum/all/give-permissions-to-files-and-folders-in-windows/78ee562c-a21f-4a32-8691-73aac1415373)
-
-Turn off UAC (User Account Control)
-
-
-## Alternative setup the efw on a Windows 10 system
-1) Download the **run_example.bat** and the **efw.exe** for your system [here](https://github.com/ComPlat/ELN_file_watcher/releases/tag/latest)
-   - In the following introduction we will assume a 64 bit Windows system.
-2) Copy the **efw_win64.exe** and the **run64_example.bat** to the target directory on your target machine
-   - In the following we use the example "C:\Program Files\file_exporter".
-3) Replace in the **run64_example.bat**:
-   - &lt;Path to efw_win64.exe&gt; with "C:\Program Files\file_exporter\"
-   - Setup all parameter (hint: use _efw_win64.exe -h_):
-   - -dst, -src, -crt, -duration, -user, -pass, -crt, -zip
-4) Create a Scheduled Task to trigger at log on.
-   - How to create a Scheduled Task:
-   1) Using the **Windows Key + R** to open run and type **taskschd.msc**. This will open Task Scheduler.
-   2) Under the actions panel, you can choose to create a back task or create a task. Click **Create Task**.
-   3) The **Create Task** screen will appear.
-   4) In the Create Task dialog, select the following:
-      1) In the **General** (tab) and select **Run with highest privileges**
-      2) In the **Triggers** (tab) press the **New** (button) and add **Begin the task, At log on**
 
 
 

diff --git a/bin/HowToSetup.md b/bin/HowToSetup.md
diff --git a/bin/efw_linux64 b/bin/efw_linux64
diff --git a/bin/efw_win386.exe b/bin/efw_win386.exe
diff --git a/bin/efw_win64.exe b/bin/efw_win64.exe
diff --git a/bin/efw_winAMD64.exe b/bin/efw_winAMD64.exe
diff --git a/bin/run386_example.bat b/bin/run386_example.bat
diff --git a/bin/run64_example.bat b/bin/run64_example.bat
diff --git a/bin/task_example.vbs b/bin/task_example.vbs
diff --git a/cmd_manager.go b/cmd_manager.go
@@ -4,6 +4,8 @@ import (
 	"flag"
 	"log"
 	"net/url"
+	"strconv"
+	"strings"
 	"time"
 )
 
@@ -12,43 +14,60 @@ type Args struct {
 	dst                  url.URL
 	duration             time.Duration
 	sendType             string
+	tType, name          string
 }
 
 // GetCmdArgs Get/Parse command line arguments manager
 func GetCmdArgs() Args {
-	var fp, dst, user, pass, crt string
+	var fp, dst, user, pass, crt, durationStr, tType, name string
 	var duration int
 	var sendType string
+	var err error
 
-	flag.StringVar(&fp, "src", "", "Source directory to be watched.")
-	flag.StringVar(&dst, "dst", "", "WebDAV destination URL. If the destination is on the lsdf, the URL should be as follows:\nhttps://os-webdav.lsdf.kit.edu/<OE>/<inst>/projects/<PROJECTNAME>/\n            <OE>-Organisationseinheit, z.B. kit.\n            <inst>-Institut-Name, z.B. ioc, scc, ikp, imk-asf etc.\n            <USERNAME>-User-Name z.B. xy1234, bs_abcd etc.\n            <PROJRCTNAME>-Projekt-Name")
-	flag.StringVar(&user, "user", "", "WebDAV user")
-	flag.StringVar(&pass, "pass", "", "WebDAV Password")
-	flag.IntVar(&duration, "duration", 300, "Duration in seconds, i.e., how long a file must not be changed before sent.")
-	flag.StringVar(&crt, "crt", "", "Path to server TLS certificate. Only needed if the server has a self signed certificate.")
+	flag.StringVar(&fp, "src", "{{ src }}", "Source directory to be watched.")
+	flag.StringVar(&name, "name", "{{ name }}", "Name of the EFW instance.")
+	flag.StringVar(&dst, "dst", "{{ dst }}", "WebDAV destination URL. If the destination is on the lsdf, the URL should be as follows:\nhttps://os-webdav.lsdf.kit.edu/<OE>/<inst>/projects/<PROJECTNAME>/\n            <OE>-Organisationseinheit, z.B. kit.\n            <inst>-Institut-Name, z.B. ioc, scc, ikp, imk-asf etc.\n            <USERNAME>-User-Name z.B. xy1234, bs_abcd etc.\n            <PROJRCTNAME>-Projekt-Name")
+	flag.StringVar(&user, "user", "{{ user }}", "WebDAV or SFTP user")
+	flag.StringVar(&pass, "pass", "{{ password }}", "WebDAV or SFTP Password")
+	flag.StringVar(&durationStr, "duration", "{{ duration }}", "Duration in seconds, i.e., how long a file must not be changed before sent.")
+	flag.StringVar(&crt, "crt", "{{ crt }}", "Path to server TLS certificate. Only needed if the server has a self signed certificate.")
 	/// Only considered if result are stored in a folder.
 	/// If zipped is set the result folder will be transferred as zip file
-	flag.StringVar(&sendType, "type", "", "Type must be 'file', 'folder' or 'zip'. The 'file' option means that each file is handled individually, the 'folder' option means that entire folders are transmitted only when all files in them are ready. The option 'zip' sends a folder zipped, only when all files in a folder are ready.")
+	flag.StringVar(&sendType, "type", "{{ type }}", "Type must be 'file', 'folder' or 'zip'. The 'file' option means that each file is handled individually, the 'folder' option means that entire folders are transmitted only when all files in them are ready. The option 'zip' sends a folder zipped, only when all files in a folder are ready.")
+	flag.StringVar(&tType, "transfer", "{{ tType }}", "Type must be 'webdav' or 'sftp'.")
 	flag.Parse()
 
+	if duration, err = strconv.Atoi(durationStr); err != nil {
+		log.Fatal("Duration must be an integer!")
+	}
+
 	if sendType != "file" && sendType != "folder" && sendType != "zip" {
 		err := "'type' has to be 'file', 'folder', or 'zip'"
-		ErrorLogger.Println(err)
+		log.Fatal(err)
+	}
+
+	if tType != "webdav" && tType != "sftp" {
+		err := "'transfer' has to be 'sftp' or 'webdav'"
 		log.Fatal(err)
 	}
 
 	if dst == "" || fp == "" || sendType == "" {
 		err := "'dst' and 'src' must not be empty!"
-		ErrorLogger.Println(err)
 		log.Fatal(err)
 	}
 
 	u, err := url.Parse(dst)
+	if (err != nil || u.Scheme == "") && tType == "sftp" {
+		u, err = url.Parse("ssh://" + dst)
+		if !strings.Contains(u.Host, ":") {
+			u.Host += ":22"
+		}
+	}
+
 	if err != nil {
-		ErrorLogger.Println(err)
 		log.Fatal(err)
 	}
 
-	return Args{src: fp, dst: *u, user: user, pass: pass, crt: crt, duration: time.Duration(duration) * time.Second, sendType: sendType}
+	return Args{src: fp, dst: *u, user: user, pass: pass, crt: crt, duration: time.Duration(duration) * time.Second, sendType: sendType, tType: tType, name: name}
 
 }
diff --git a/docker-compose.yml b/docker-compose.yml
diff --git a/go.mod b/go.mod
@@ -1,3 +1,14 @@
 module ELN_file_watcher
 
-go 1.10
+go 1.19
+
+require (
+	github.com/StarmanMartin/gowebdav v0.0.0-20220901075112-8721ee532c0c
+	github.com/pkg/sftp v1.13.5
+	golang.org/x/crypto v0.1.0
+)
+
+require (
+	github.com/kr/fs v0.1.0 // indirect
+	golang.org/x/sys v0.1.0 // indirect
+)
diff --git a/go.sum b/go.sum
@@ -0,0 +1,39 @@
+github.com/StarmanMartin/gowebdav v0.0.0-20220901075112-8721ee532c0c h1:9fIDkeXKjWFobi22q8VVbMLQdCwKKbsj6X5uc94lP4E=
+github.com/StarmanMartin/gowebdav v0.0.0-20220901075112-8721ee532c0c/go.mod h1:Ey6vSiGSv4PoRe+kgLsOtCBKkCNbOsDOaLt4AqMHboc=
+github.com/davecgh/go-spew v1.1.0 h1:ZDRjVQ15GmhC3fiQ8ni8+OwkZQO4DARzQgrnXU1Liz8=
+github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/helloyi/go-sshclient v1.1.1 h1:yRcrc/Q1nJ2hYbtVFfBBTyneQLqr2vy/jD3/GneyirI=
+github.com/helloyi/go-sshclient v1.1.1/go.mod h1:NrhRWsYJDjoQXTDWZ4YtVk84wZ4LK3NSM9jD2TZDAm8=
+github.com/kr/fs v0.1.0 h1:Jskdu9ieNAYnjxsi0LbQp1ulIKZV1LAFgK1tWhpZgl8=
+github.com/kr/fs v0.1.0/go.mod h1:FFnZGqtBN9Gxj7eW1uZ42v5BccTP0vu6NEaFoC2HwRg=
+github.com/pkg/sftp v1.13.5 h1:a3RLUqkyjYRtBTZJZ1VRrKbN3zhuPLlUc3sphVz81go=
+github.com/pkg/sftp v1.13.5/go.mod h1:wHDZ0IZX6JcBYRK1TH9bcVq8G7TLpVHYIGJRFnmPfxg=
+github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
+github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
+github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
+github.com/stretchr/testify v1.7.0 h1:nwc3DEeHmmLAfoZucVR881uASk0Mfjw8xYJ99tb5CcY=
+github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
+golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
+golang.org/x/crypto v0.0.0-20201221181555-eec23a3978ad/go.mod h1:jdWPYTVW3xRLrWPugEBEK3UY2ZEsg3UU495nc5E+M+I=
+golang.org/x/crypto v0.0.0-20211215153901-e495a2d5b3d3/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
+golang.org/x/crypto v0.1.0 h1:MDRAIl0xIo9Io2xV565hzXHw3zVseKrJKodhohM5CjU=
+golang.org/x/crypto v0.1.0/go.mod h1:RecgLatLF4+eUMCP1PoPZQb+cVrJcOPbHkTkbkB9sbw=
+golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
+golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
+golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.1.0 h1:kunALQeHf1/185U1i0GOB/fy1IPRDDpuoOOqRReG57U=
+golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw=
+golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
+golang.org/x/term v0.1.0 h1:g6Z6vPFA9dYBAF7DWcH6sCcOntplXsDKcliusYijMlw=
+golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
+golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
+golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
+gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
+gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c h1:dUUwHk2QECo/6vqA44rthZ8ie2QXMNeKRTHCNY2nXvo=
+gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=