Move to XenForo software

[pwd]

Hi Nep and other sys admins,

just wondering if you have a preview etc... of what the site may look like on XenForo is the template the same as seen here?https://xenforo.com/community/

This is just personal preference but I tend to favor the compactness of the current phpbb template with fairly compact text and boarders etc... I find many modern forums very bloated and too chunky. I realize this is to accommodate devices with different sized displays for mobile/tablets etc... and give a responsive template. If we had the ability to select a template with different styling (like desktop vs mobile) that might be a nice thing to have.

[neptronix]

Hey PWD, i am also a fan of high density looks.
In PHPBB, the theme was actually too dense, and i had to add more spacing in order for it to work on mobile and look good on modern hi-res screens.

2021-06-15 08_59_58-XenForo - Brave.png

In xenforo, i have to move things in the opposite direction, but not by a ton.
This is the kind of density i'd aim for:

2021-06-15 09_27_52-Large EV General Discussion _ XenForo - Brave.png

This is identical to what we have on ES currently. And it's possible to push a bit further.

I'd also add a bit more contrasting colors in certain areas. Xenforo is a bit too subdued in some places and this makes it harder to read for people with vision issues. It can also look sort of drab.

Xenforo does not need as much CSS work as PHPBB did. It also has a better HTML structure, making it less of a fight to work with.

I'm open to design feedback when we have a proper tester with all the database transformation and bugs kicked out.

[pwd]

Nice, that preview looks pretty good to me and seems compact enough for a desktop display etc... Glad to here it doesn't need as much CSS work. I'm sure it is using much newer HTML standards too. Thanks for the screenshots.

[neptronix]

Yeah, it's a good balance between mobile/tablet clickability and making good use of desktop space.

Here's the 'design language' i've been working on lately:

officemanual-2.0.png

contractormate-mockup-v2.85.png

As you can see, it's less colorful in general.. but the colors are more meaningful and stand out more.

How does this look on your eyes and how do you feel about some variant of this design being used for ES?

[nicobie]

Any news on the switch to XenForeo?

[neptronix]

Thanks for asking, i was thinking i should update this thread.

We do have an initial Xenforo transition done and most of the bugs are noted but it could use a further look through so we can have a FULL list of what needs tackling before the final launch. i'd be willing to hand the bug finding job to anyone with a discerning eye.

The list of buggy/missing data after a Xenforo move is quite daunting to fix though, honestly we're just discouraged at the moment!

ES also needs some major re-engineering of it's hosting and backup system so that it can have adequate space for expansion and long term backup, but in a cost effective way.. i've just started experimenting with that setup for other clients before ES gets the treatment.

So we're waiting on:
+ The server re-engineering to happen, which should be in a few weeks at most since i've just started doing it across the board for all servers i command.
+ Me to have a big chunk of time to write processes to fix the buggy data; probably looking at fall realistically.

If anyone here knows some decent PHP or SQL-database fu and is willing to volunteer, and some free time.. i'd be happy to delegate some of that job out so we can get to Xenforo faster.

[nicobie]

Thanks for a the update.

When the forum went screwy today is what made me think of it.

BIG thanks to whoever fixed it.

[neptronix]

I fixed it and the cause is the big server disk problem we're running up against

I've stuffed a bandaid on the problem for now that will last a couple weeks but the server job needs to get out of the way sooner than later. Getting that out of the way clears a hurdle

[neptronix]

FYI i have put this week aside to complete this. Finally.
I have a sensitive information censored virtual machine copy of the server which will be handed to any volunteers who want to aid in the work.

I have also lowered the bar for launch in order to simply get it done.
All the extensions that give Xenforo even nicer functionality than stock will be added later.
Customization of the Xenforo theme will also happen later.

Even without the sparkle and shine, Xenforo makes for a nicer board.

I'll give 'yall a link to a preview once things are settled.

[neptronix]

Guess i don't need to script this. Xenforo allows me to put together a translation from phpbb smiley to xenforo smiley in the interface by allowing multiple codes.
VERY COOL

2021-08-17 15_55_19-(61) phpbb xenforo emoji translation test post - Endless Sphere.png

Well there's one big task done.

[kauaicycler]

FYI i have put this week aside to complete this. Finally.
I have a sensitive information censored virtual machine copy of the server which will be handed to any volunteers who want to aid in the work.

I have also lowered the bar for launch in order to simply get it done.
All the extensions that give Xenforo even nicer functionality than stock will be added later.
Customization of the Xenforo theme will also happen later.

Even without the sparkle and shine, Xenforo makes for a nicer board.

I'll give 'yall a link to a preview once things are settled.

I'm willing to help.

[neptronix]

Cool. The one big blockade we have is that various types of former youtube embed tags come in different formats. These need to be unified into similar tags and then transformed into xenforo's [media] tags.

This will require a strong SQL-fu, maybe PHP-fu and/or regex-fu.

And needs to happen on a local dev virtual machine with at least 120gb of free space. And a good chunk of ram ( 4gb? ) and CPU to process the 7 gigabytes of text data in our database. It will take a VERY long time ( prob hours ) to process.

Is that something you're up for?

[kauaicycler]

Cool. The one big blockade we have is that various types of former youtube embed tags come in different formats. These need to be unified into similar tags and then transformed into xenforo's [media] tags.

This will require a strong SQL-fu, maybe PHP-fu and/or regex-fu.

And needs to happen on a local dev virtual machine with at least 120gb of free space. And a good chunk of ram ( 4gb? ) and CPU to process the 7 gigabytes of text data in our database. It will take a VERY long time ( prob hours ) to process.

Is that something you're up for?

Unfortunately I don't have much time available tomorrow as it's my birthday. Is this something that can be started on the 19th? I'm EST and work mostly daytime hours. I have a Macbook Pro with 16gb ram. The LAMP stack powered all of my sites although nowadays I've been using Javascript for my personal projects.

[neptronix]

Cool. Glad you know LAMP
Yeah this can wait until the 19th.
Tomorrow i'll work on finishing testing the image recompressor. So you may get a VM that's smaller than 120gb hopefully.

16gb of ram might work; at a bare minimum, ES can probably do all the calculations on 4gb of system memory if you manage the memory well in your script.

Yeah write in whatever language you prefer, as long as the output is a script i can run before hitting the big red button on migration

Happy birthday and let's get together on the 19th.

[kauaicycler]

Great! I'm a big fan of vagrant and virtual box for creating VM's with ansible to provision. Send me what you have when your ready so I can get my dev environment setup.

[kauaicycler]

A few past projects https://portfolio.kahunacoder.com Nothing as big as ES though.

[neptronix]

I like your resume. I'd love to give you a chance to contribute. Will schedule a call with you tomorrow.

I'm compressing the virtual machine and compiling instructions on how to set it up as we speak.

[nicobie]

Good luck guys! Wish I had the skills to help.

God speed...

[neptronix]

Not many people have the skills! Got two people on the line and hoping to get some contribution

Here is the current list of tasks:

es tasks.png

What we could use is users willing to test out our 'first draft' of ES on Xenforo. I believe i have a few volunteers signed up already. The feedback would be extremely valuable. It would be a good thing to get a set of differing opinions and eyes on it... for sure!!!

If i can scratch up some time next week, we could do a more complete initial launch with all of the plugins configured and tested. That's the one step we can leave out and implement later if that doesn't happen.

[nicobie]

Well I can do that. You just need me to spend some time cruising the new site checking how it works? No problem. Sign me up.

Nick

[neptronix]

Yup that's all we need. Basic user testing, and a lot of it.

I have 3 volunteers going now. kauaicycler is trying to find us a wiki replacement. AdamW and comradeM are on code.

[neptronix]

Finally got back to debugging the image recompressor.

2021-08-25 19_26_47-BashTOP.png

God only knows how long this is going to take on the first run of 55gb of images.

Will see results in 24-48 hrs?

[neptronix]

Bah, test failed due to one line in the code.
My brain is also shutting down due to the California smoke re-entering my state and choking me out.

Hopefully next week the air clears and i can regain the lost momentum.

[neptronix]

The debugging of the image processor is halfway done. The batch logic is now working 100%.

2021-08-31 15_34_23-https___192.168.1.2_admintools_endless-tools_ESCompressor - Brave.png

There were 6279 corrupt images out of 216480, which is 2.9% of our images in total.

Using irfanview's batch convertor, i was able to recover 354 of those images, which are in various states of repair, as you can see.

These look very old and were likely from a corruption episode very early in the forum's life.

The windows explorer thumbnailer produces small previews of up at least 1000 of these images but nothing i have used can read them. If the explorer thumbnailer can show me a preview, i believe there may be some tool that can recover these jpegs.

I'm going to try some other software to run a batch recovery process. Irfanview is probably not the ideal tool despite having it's own routines for reading jpegs etc that may be more lax.

I'll see what percentage of the images i can repair and report back.

Now on to debugging the image crunching aspect of our processor itself.

[neptronix]

All bugs finally kicked out. The hardest part is done.

2021-08-31 19_58_07-https___192.168.1.2_admintools_endless-tools_ESCompressor - Brave.png

Looks like the logic is good. It leaves corrupt files alone. If a file is larger after recompression with mozjpeg ( usually the case if someone encoded it with uber low quality setings), it leaves it alone also so we don't lose image quality. It also knows to not touch something that isn't an image and doesn't make an assumption that the file extension is right.

I'm running a sample size of 10,000 files overnight which will take some hours. I'll have results in the morn.
mozjpeg and pngquant are CPU hogs and it doesn't help that i have the compression levels as high as they can go. The actual process on the live server will probably a week.

I expect a 66% reduction of our file store based on previous tests. Should shave a big portion off the phpbb to xenforo conversion time, which is something like 16 hours.

I'm also going to do a test see if imagemagick can read a higher portion of the images. Hopefully it doesn't use the same jpeg reading libraries, as jpegs are where we consistently see the most corruption.