Re: linkdups

[Steven Pritchard <steve@silug.org>]

On Wed, Mar 09, 2022 at 10:19:15PM -0600, Steven Pritchard wrote:
> Now that I have it working, I need to put some effort into making it
> work quickly.  At the moment it's pretty slow, but it should
> eventually be as fast or faster than the old version with some work.

I rewrote the function that shoves data in the database, and now that
part runs around 76x faster on my test box.  :-D

If anyone is interested, the original code was doing a SELECT to see if
there was already an entry for a file in the database, then did either
an UPDATE or INSERT as appropriate.  Now there is a UNIQUE INDEX and a
TRIGGER combined with a blind INSERT that has an ON CONFLICT clause.
Basically all of the logic moved to the database.

  https://github.com/silug/linkdups/commit/e478f7ff592d8e486d8b0b1d2ded7a964b60dc9e

(And yes, I'm learning a lot of SQL here...)

Doc, if you tried what I had published last night, definitely try it
again.  I had to make a small tweak to the database, but it should
upgrade the database for you automatically.

-
To unsubscribe, send email to majordomo@silug.org with
"unsubscribe silug-discuss" in the body.