Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Teleporter does not overwrite Domains #2518

Open
JvdMaat opened this issue Jan 31, 2023 · 5 comments
Open

Teleporter does not overwrite Domains #2518

JvdMaat opened this issue Jan 31, 2023 · 5 comments

Comments

@JvdMaat
Copy link

JvdMaat commented Jan 31, 2023

Versions

  • Docker 2022.12
  • Pi-hole: 5.14.2
  • AdminLTE: 5.18
  • FTL: 5.20

Platform

  • OS and version: Ubuntu 22.04.1 LTS
  • Platform: Docker

Expected behavior

I have two piholes. I removed and added Domains (whitelist/blacklist) from one, and executed a teleporter export. On the second pihole which had the same Domains (white/blacklists) prior, I executed a teleporter import with all checkboxes checked except Audit Log. Expected the Domains to have been updated to what was on the other pihole since the "Clear existing data" checkbox was checked.

Actual behavior / bug

Domains do not get updated correctly. New domains do get added. Domains that were removed do not get removed on target. Domains that were changed from Status Enabled to Disabled do not update on target.

Initially after searching here, it was hinted this could be a locale issue. Docker runs in POSIX locale, while the source pihole (Raspberry Pi Zero W) runs UTF-8.
I changed the source to POSIX as well to have a level playing field, however this did not change the outcome.

Debug Token

@yubiuser
Copy link
Member

yubiuser commented Feb 2, 2023

I can reproduce this. It's fairly easy: create a backup, make some changes to the domainlist and restore the backup. For me, deleted entries got re-added (this is Ok), state (enables/disabled) did not change after import (not Ok), newly added domains lost group assignment (not OK).

@yubiuser yubiuser transferred this issue from pi-hole/pi-hole Feb 2, 2023
@github-actions
Copy link
Contributor

github-actions bot commented Mar 5, 2023

This issue is stale because it has been open 30 days with no activity. Please comment or update this issue or it will be closed in 5 days.

@github-actions github-actions bot added the stale label Mar 5, 2023
@yubiuser yubiuser added Bug and removed stale labels Mar 5, 2023
@csm10495
Copy link

I'm hitting this as well. If I delete a domain in one then teleport it to the other, the domain still remains.

@gsgleason
Copy link

gsgleason commented Sep 11, 2024

I've been troubleshooting the issue of an import not actually removing domains that do not appear in the archive being imported while the clear existing data option is selected.

Here is what I've found:

For domains, the POST request has some values that correspond to the checkbox inputs on the teleporter page. While processing the POST request, these get passed to archive_restore_table(), with possible values of 'table' as blacklist, regex_blacklist, whitelist, and regex_whitelist.

Line 91 of teleporter.php (https://github.com/pi-hole/web/blob/master/scripts/pi-hole/php/teleporter.php#L91) is the part of archive_restore_table() that will flush the tables, if requested, but since blacklist, regex_blacklist, whitelist, and regex_whitelist are not actually table names in the gravity database, no flush is performed.

What's interesting is that there is another function, archive_insert_into_table() , which handles text files in the archive (as opposed to json files). I'm guessing this is for an older version. This has a section starting on line 241 that will set $table to "domainlist" for whitelist, blacklist, and regex_blacklist (It's missing regex_whitelist for some reason; maybe it didn't exist for the version that used plain text files for the domain lists). When it gets to the part that flushes tables, it correctly flushes the domainlist table prior to inserting the new data.

What's also weird is that the archive_insert_into_table() uses another function flush_table(), whereas archive_restore_table() has it's own logic for flushing the requested table. I don't see why there are two places to do the same thing, and I don't understand why there are two functions to import the data depending on the file type.

This could be fixed many different ways; domainlist needs to be flushed based on the type of entry, depending on which domain list is being updated.

Here was my solution:

gsg@apu1d4:/var/www/html/admin/scripts/pi-hole/php$ diff teleporter.php teleporter.bak
135,136c135
<             $type = LISTTYPE_WHITELIST;
< 	   if ($flush) flush_table('domainlist', $type);
---
>             $type = 0;
138,139c137
<             $type = LISTTYPE_BLACKLIST;
< 	   if ($flush) flush_table('domainlist', $type);
---
>             $type = 1;
141,142c139
<             $type = LISTTYPE_REGEX_WHITELIST;
< 	   if ($flush) flush_table('domainlist', $type);
---
>             $type = 2;
144,145c141
<             $type = LISTTYPE_REGEX_BLACKLIST;
< 	   if ($flush) flush_table('domainlist', $type);
---
>             $type = 3;

I would rather refactor the two functions to both use flush_table(), but I feel like this is an adequate proof-of-concept.

@Exit24
Copy link

Exit24 commented Oct 8, 2024

I'm having a similar issue using Pi-hole v5.18.3 FTL v5.25.2 Web Interface v5.21.

I have 2 piholes. I added a whitelist entry on pihole 1, and then I ran a teleporter backup. On pihole 2 I executed a teleporter import with all checkboxes checked. The new whitelist entry on pihole 1 doesn't appear on pihole 2 after the teleporter import.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants