In the "AI image search" view images can be searched by text after AI analysis of images. The images found are displayed in the same way as in the particular browsing view and can be drag'n'dropped for use in writings.
Before images can be searched using text they must first be analysed by an AI, of which there are several options to choose from. The AI analysis per image is at this stage intended to work in such a way that there is no need to send the images to another service and instead all the analysis is done on the server(s) that the client may already be using. This may impose limitations on how quickly images can be analysed, as some AI models require a lot of computational power from the processor.
Some AI models may take tens of seconds to analyse a few hundred images, others minutes. The results of image analysis are automatically stored in a database so that they are available for comparison when retrieving images using text queries.
The analyse results of the different AI models are not compatible with each other, but the user interface has been designed in a way that it provides clear indicators of which AI model has been used for which image containers.
After the first image analyses, the images can already be searched using text queries and if more images are added to image containers later, a clear indicator of the missing image analyses is presented, so one could know which image container the image analysis should be applied to. Background image analysis would also be an implementation option, but it has its risks, e.g. in terms of excessive server load.
Preliminary experiments show that the search results are pleasantly reliable, e.g. when searching for "city streets", the search results do match the query. The analysis results can be deleted per image container and per AI by Ctrl-clicking on the name of the AI model.
The results of the image analyses are stored in the same database management system that is already used for storing data such as writing related data, i.e. MariaDB. Newer versions of the database have the possibility of using vector data, which is very suitable for the purpose. In the past, the use of a separate vector database (e.g. Qdrant, Pinecone or Weavite) had been considered.
To be comtemplated later is to consider, if analysing of images could/should be done in a separate service with a large number of computing resources available. Currently, it is possibly to increase performance by directing computationally intensive operations away from a virtual server and toward e.g. a dedicated server. However, it seems that a single virtual server is rather sufficient to momentarily utilize an AI model without e.g. running out of memory on the server. Admittedly, currently AI-based image analysis uses images in their 360 pixels wide version. When using much larger ones, all the resources on a server would run out of quite quickly.
The results of AI image analysis are not stored in project backups, so if a project and its image catalogs are deleted, the related image analysis results would also be gone.
<-- Appealing reader with well-choosen writing names and mainimages
Screenshots from a tablet device to quick saves -->