Remember rejected entries

Hello.

I’m trying to have Flexget remember once rejected entries and to not accept then again in subsequent tasks (and in future runs).

The feed for the first task (partial-feed) has a subset of the entries of the second task (full-feed). So some of the entries appear again in full-feed. For testing purpose, please have a look at the following config: (All executed with --test using version 3.1.67)

tasks:

  partial-feed:
    rss:
      url: https://nyaa.si/?page=rss&u=HorribleSubs
      all_entries: no
    quality: '1080p+'
    regexp:
      accept:
        - HorribleSubs

  full-feed:
    rss:
      url: https://nyaa.si/?page=rss&c=1_2&f=1
      all_entries: no
    regexp:
      accept:
        - HorribleSubs

The (abridged) result looks like this:

...
2000-00-00 00:00:00 VERBOSE  task          partial-feed    REJECTED: `[HorribleSubs] Titel [720p].mkv` by quality plugin because 720p does not match quality requirement [<Requirements(1080p+)>]
2000-00-00 00:00:00 VERBOSE  task          partial-feed    ACCEPTED: `[HorribleSubs] Titel [1080p].mkv` by regexp plugin because regexp 'HorribleSubs' matched field 'title'
...
2000-00-00 00:00:00 VERBOSE  task          full-feed       ACCEPTED: `[HorribleSubs] Titel [720p].mkv` by regexp plugin because regexp 'HorribleSubs' matched field 'title'
...

I would have expected that full-feed would reject the 720p-version again, at least that is how I understood remember_rejected should behave.

I can force my expected behavior with:

tasks:

  partial-feed:
    rss:
      url: https://nyaa.si/?page=rss&u=HorribleSubs
      all_entries: no
    quality: '1080p+'
    regexp:
      accept:
        - HorribleSubs
# --- diff start
    digest:
      list: digest_rejected
      state:
        - rejected

  mark-as-seen:
    from_digest:
      list: digest_rejected
    accept_all: yes
# --- diff end

  full-feed:
    rss:
      url: https://nyaa.si/?page=rss&c=1_2&f=1
      all_entries: no
    regexp:
      accept:
        - HorribleSubs

And the (abridged) result looks like this:

...
2000-00-00 00:00:00 VERBOSE  task          partial-feed    REJECTED: `[HorribleSubs] Titel [720p].mkv` by quality plugin because 720p does not match quality requirement [<Requirements(1080p+)>]
2000-00-00 00:00:00 VERBOSE  task          partial-feed    ACCEPTED: `[HorribleSubs] Titel [1080p].mkv` by regexp plugin because regexp 'HorribleSubs' matched field 'title'
...
2000-00-00 00:00:00 VERBOSE  task          mark-as-seen    ACCEPTED: `[HorribleSubs] Titel [720p].mkv` by accept_all plugin
...
2000-00-00 00:00:00 VERBOSE  task          full-feed       REJECTED: `[HorribleSubs] Titel [1080p].mkv` by seen plugin because entry with title `[HorribleSubs] Titel [1080p].mkv` is already marked seen in the task partial-feed at 2000-00-00 00:00:00
2000-00-00 00:00:00 VERBOSE  task          full-feed       REJECTED: `[HorribleSubs] Titel [720p].mkv` by seen plugin because entry with title `[HorribleSubs] Titel [720p].mkv` is already marked seen in the task mark-as-seen at 2000-00-00 00:00
...

In the above the task mark-as-seen “accepts” all rejected entries of partial-feed and that way they get marked as seen and not accepted again. I also tried to crate an entry_list containing the rejected items, but all I could archive where more lines with no better effect then what you can find in the second config. (My YAML/Jinja2 skill are just are not high enough yet…)

It works, but it doesn’t look right to me. Is there no better / more elegant way to solve this?
Is this the expected behavior of quality? Do I miss some plugin/setting that I have to use?

Thank you for your feedback.


Side note: I tried the series plugin, but it has a hard time with this type of entries, where the names are not very “conform”. So I mostly switched to the trusted regexp, which works for me. And yes I know there is the nyaa plugin, but for my purposes it is to limited (and I solve it by using pre-filted feeds).

Since it doesn’t have a quality requirement it will get accepted always by the regex. You can have multiple feeds in a task like this:

tasks:
  task-rss:
    quality: 1080p+
    regexp:
      accept:
        - HorribleSubs
    inputs:
      - rss: 
          url: https://nyaa.si/?page=rss&c=1_2&f=1
          all_entries: no
      - rss: 
          url: https://nyaa.si/?page=rss&u=HorribleSubs
          all_entries: no

Quality plugin works as it should (as seen in your partial-feed task – it rejects the 720p and accepts 1080p). Even if you would add quality to your full-feed and your partial feed has already downloaded [HorribleSubs] Titel [1080p] if the full-feed gets [HorribleSubs] Titel [2160p] it will download it :smile:. As you discoverd with your workaround you need to ‘see’ also the rejected entries so the seen plugin will not accept them in following tasks.

The elegant solution would be to use the series plugin and try to change the internal parser to guessit maybe you will have better results

also since you anyways only accept releases from HorribleSubs why not combine the two urls into one https://nyaa.si/?page=rss&c=1_2&f=1&u=HorribleSubs ?

Thank you for your feedback @putyn. The reason I don’t use quality in full-feed is because not all series in there have quality information that could be parsed and then would be rejected (even if I wanted to accept them). That could be solved by using assume_quality: any. But then I would reject series that only have 720p with no 1080p if I use quality: '1080p+' in combination with that…
The best solution probably would be to use the series plugin. I will try guessit and see if it works for me. That way I could use timeframe, upgrade and similar configs and don’t have these problems.

The reason why I don’t combine the inputs is because the above is just a boiled down example to the specific problem. The full config looks more like this: (Along with more tasks and global template.)

tasks:
...
  horriblesubs:
    inputs:
     # main tracker
      - rss:
          url: https://nyaa.si/?page=rss&u=HorribleSubs&q=1080p
          all_entries: no
          other_fields:
            - { 'nyaa:infoHash': torrent_info_hash }
      # backup feed
      - rss:
          url: https://www.tokyotosho.info/rss.php?username=hs
          all_entries: no
      # Magnet feed if tracker fail
      - rss:
          url: http://www.horriblesubs.info/rss.php?res=1080
          all_entries: no
    quality: '1080p+'
    # use RegEx to accept and reject series
    include:
      - horriblesubs.regexp.yml
  ...
  anime:
    inputs:
      - rss:
          url: https://nyaa.si/?page=rss&c=1_2&f=1
          all_entries: no
          other_fields:
            - { 'nyaa:infoHash': torrent_info_hash }
      - rss:
          url: https://anidex.info/rss/?cat=1&lang_id=1&r=1
          all_entries: no
      - rss:
          url: https://www.tokyotosho.info/rss.php?filter=1&entries=750
          all_entries: no
    include:
      - global.regexp.yml
      - anime.regexp.yml
  ...
...

As you can see, I make use of the input plugin to make preferences to where to get the torrent from and add some resilience.

I tired guessit but sadly it wasn’t possible to tag all entires, they are just too hard to parse. So I’m back to regexp.