Scienvera Writeup Part 1
Starting Off
This was my fun project, it did not have a real-life plan. Because of that, I have decided to code this with C++. Why? No freaking idea, just wanted to practice?
But this was not viable. Because of the XML and JSON data files that were huge, i had a hard time parsing the data into it. I mean yeah we can fix that. But here is the second hard thing.
Freaking AI/ML/whatever you call it. I did not have a beast machine, so it can not run 3B+ models. Because of that, i can not use reasoning function. Sure, i can offload that to OpenAI or something, but price will be too high for fun project. Because of that, I only did a prototype with hugging face.
When i demonstrated this to my colleagues and several professors, I realised that people can actually use it. From this now on, I had 2 choices (well besides private usage):
- I can continue to create this with C++ or Python, and embed a licensing system and sell it that way.
- Or i can completely scrap everything i know, and maybe.. I can create a website? FINALLY SOME SPRING ACTION!
For the first option, distribution and licensing will be difficult. Plus, you never know the possibilities with local program. Someones gonna try to crack it you know. And i wont go into the digital signing shit, will take so long..
For the second option, i have started my journey with Java, but i never fully understood Spring in middle and high school, so i always left it to learn later. But there is good chance now.
My Previous Plan
Well, this was my original plan. Because i thought it was not a big project(WHAT?), so i was decided to keep it simple.
Front-End:
- Probably use React, or Angular idk. Some framework
Back-End
- 1 Spring Boot monolith to contain all of functions.
My Spring Journey
Well, it was freaking rough. I really dont like to learn with online courses. Rather I cant learn. Because i want the every edge case, every possible errors that you can do etc.. simply impossible. So, i started reading the whole spring documentation.
My project consisted with only 1 monolith structure. It had basic Rest API principles. I started to learn about actual TCP/IP protocols, HTTProtocols(how can you even say that lol).
After a month, i finally created a Spring Boot application that can search through PubMed and other services. In this time, I found out Google Scholar does not allow this type of data searching, yikes. Its fine, PubMed will do more than enough. But there is one key part missing. FREAKING AI ML THING AGAIN.
Similarity Checking: AI, ML? Whatever was this again?
Yeah, I was really confused. I started really late to this machine learning stuff, i even learned LLMs like a year later after their popularity increase. Yes I know about how it works, but how much of us can actually use it besides brainlessly smashing keyboard to ChatGPT to give results?
So, my options were clear. First, i can outsource it. That’s really easy, but will cost me so much. Even if i was planning it to go public, pricing will be skyrocket.
Because our method gives X amount of articles, if total size is <= X. That means if i request 2000, and if total size of retrieved articles is 1500, it will give 1500. HOW CAN I EVEN GIVE THAT MUCH INFO TO WRAPPER?
So, outsource is not an option. What are my choices? Oh yeah.. local chat and ask it. Well not gonna cut it. Because it cant really go that deep with contexts, and if i want to go that route, i need a huge LLM capable machine. Renting these are so high, let alone my machine is not really for GPU intensive stuff.
So, days of searching, and i have found something.
Embedding Models
Why didn’t i think of that? I mean all the words that we use gets tokenized and will be stored as vectors. And similarity check is similar (such irony) to cosine angle check between vectors, so why not use it?
So, I started how can I use embedding models with “local machine” and near 0 gpu usage.
For this, I have used Spring AI libraries. I have implemented, but guess what? We need a database! But small issue, vectors cant be stored in every database. I wanted to use good old MySQL.. But realised the easiest and effective way for that is PostgreSQL, with vector extension. Another new thing. great.
Messing with Ollama & Embedding
This shit was not easy for relatively Spring beginner. Lots of trial and error (yes i still hate online courses), trying to learn PostgreSQL, and integrating everything.
So, 2 sleepless weeks later, i was finally be able to implement this into my research articles. Great!
Summary Until Now
I have spent 1.5 months to create C++ prototype, a fully working Spring Boot application that uses text embedding model with ollama to cross cosine check my article with all related X amount of articles that are retreived from pubmed. All good. But.. what about user interface? What about users? And most importantly, what about security?
Now onto the fun part.
Okay, now this is the most complex part, i have to cut short.
I wanted to start with user interface, why not. what, whats the problem? Oh. I have no fucking idea about React. great! With this task, i can not thank Mantine UI for creating such a simple kit for rapid prototyping. I mean i dont need to know actual “react backend” stuff, because only thing i need is a simple API call to Spring and that is it.
Because of that, I somehow grasped the “designing” part of react, and started to create.
User Interface
The user interface will have a simple design that has separate tab for research topic & your source, and another tab for parameters such as article release date etc..
After 1 week, i finally created a prototype for reference search and yeah it was working fine. Not adding the extra 1 week for figuring out the connection between the two 😀
Now… onto “the beast.“
HAHA SPRING SECURITY GO BRRRRRRRRRRRRRRRRRRRRR
This was hell. Literal suffering. There was so much concepts behind the scenes and you tell me there was a difference between “Authorization” and “Authentication”? What the hell is Token? JWT? OAuth? What??
Alright, enough ranting. This part was even more complicated than AI thing. But i have learned so much. My dumbass couldnt figure out the oauth, so decided to go plain old JWT token generation. Yeah oauth protocol uses JWT too, but I was manually designing and creating it.
So, while i was trying to experiment with basic authentication(adding username and password to every request), I started to learn about JPA and Hibernate.
In a week, I have managed to create a “basic” authentication fully working. I have created a simple login page for it too (it was useless lmao i was passing credentials every request).
What? Giving password for every request??
So that is where i learned JWT stuff. It essentially generates a token that can be verified using cryptographic functions so once its generated, you can not modify it. Just remember that sentence, will come soon. With this, I started to actually creating a database structure, Role and Authority system and replacing “basic” authentication to JWT bearer authentication model.
This allowed me to actually use the login page now :D. And i can store the token with redux-persist so win-win.
Do you see the huge security flaw? I didn’t. And so I moved on without actually assessing the situation.
So.. it was done?
I mean the project was done, 2 months. One monolith spring application, one simple react site. Invitation only(i mean manual lol) research tool. But thats where it struck me I could.. market it?
The Literal Hell Begins.
So. If you managed to read that, huge props to you. This was too long i cut it into 2 parts. If you want to see my flaw and total rewrites etc.. you can read my part 2 here.