Compare commits
No commits in common. "a1f7365c3ff0cb0f5846a2f6c641db84297e8aaa" and "15d4de3b45423cef3b96414584809865408fb156" have entirely different histories.
a1f7365c3f
...
15d4de3b45
@ -1,120 +0,0 @@
|
|||||||
\section{Service Design} \label{sec:sd}
|
|
||||||
This section will discuss the design of the service.
|
|
||||||
The design on this section is an ideal design solution, where no time limitations or engineering limitations were considered.
|
|
||||||
This section tries to provide a description of a designed solution that would allow for the best user experience possible.
|
|
||||||
|
|
||||||
The design proposed in this section can be viewed as a scoped version of this project, and the \hyperref[sec:si]{Service Implementation} section will discuss how the scope was limited so that the service would achieve the primary goals of the project while following the design, within the time frame of this project.
|
|
||||||
|
|
||||||
\subsection{Structure of the Service}
|
|
||||||
|
|
||||||
The service is designed to be a 4 tier structure:
|
|
||||||
\begin{itemize}
|
|
||||||
\item{Presentation Layer}
|
|
||||||
\item{API Layer}
|
|
||||||
\item{Worker Layer}
|
|
||||||
\item{Data Layer}
|
|
||||||
\end{itemize}
|
|
||||||
|
|
||||||
This structure was selected because it allows separation of concerns.
|
|
||||||
The layers were separated based on what resources are required by that layer.
|
|
||||||
|
|
||||||
The presentation layer requires interactivity of the user, and therefore it needs to be accessible from the outside, and be simple to use.
|
|
||||||
The presentation layer was limited from being any interaction method to be a web page.
|
|
||||||
The web page can a separate server, or as part of the main API application, if it is in the same.
|
|
||||||
|
|
||||||
The API layer, is one of the most important parts of the service. As it's going to be the most used way to interact with the service.
|
|
||||||
The user can use the API to control their entire model process from importing, to classification of images.
|
|
||||||
|
|
||||||
The Worker layer, consists of a set of servers available to perform GPU loads.
|
|
||||||
|
|
||||||
The Data layer, consists of stored images, models, and user data.
|
|
||||||
|
|
||||||
|
|
||||||
\subsection{Interacting with the service}
|
|
||||||
As a software platform, this project requires a way for users to interact with the service.
|
|
||||||
This interface is mainly intended to be as a control and monitoring interface.
|
|
||||||
The user would use the interface to set up and manage the models, then most of the interactions would happen via the API.
|
|
||||||
|
|
||||||
While there were no specific restrictions on what the interface can be, it makes most sense for it to be a web page.
|
|
||||||
This is because most software as a service applications are controlled with web pages, and the API is already a web-based application.
|
|
||||||
|
|
||||||
Independently of the kind of the application is, it needs to allow users to fully control their data in an easy to use and understand way.
|
|
||||||
The application should allow users to:
|
|
||||||
%TODO add more
|
|
||||||
\begin{multicols}{2}
|
|
||||||
\begin{itemize}
|
|
||||||
\item Manage access tokens.
|
|
||||||
\item Upload images for training.
|
|
||||||
\item Delete images.
|
|
||||||
\item Request training model.
|
|
||||||
\item Delete models.
|
|
||||||
\item Classify Images.
|
|
||||||
\item See previous classification results
|
|
||||||
\item Keep track of model accuracy
|
|
||||||
\end{itemize}
|
|
||||||
\end{multicols}
|
|
||||||
|
|
||||||
Aside from being able to perform the above tasks, there are no restrictions on how the application needs to be architected.
|
|
||||||
|
|
||||||
\subsection{API}
|
|
||||||
As a software as a service, one of the main requirements is to be able to communicate with other services.
|
|
||||||
The API provides the simplest way for other services to interact with this service.
|
|
||||||
|
|
||||||
The API needs to be able to perform all the tasks that the application can do, which include:
|
|
||||||
% TODO maybe remove
|
|
||||||
\begin{multicols}{2}
|
|
||||||
\begin{itemize}
|
|
||||||
\item Manage access tokens.
|
|
||||||
\item Upload images for training.
|
|
||||||
\item Delete images.
|
|
||||||
\item Request training model.
|
|
||||||
\item Delete models.
|
|
||||||
\item Classify Images.
|
|
||||||
\item See previous classification results
|
|
||||||
\item Keep track of model accuracy
|
|
||||||
\end{itemize}
|
|
||||||
\end{multicols}
|
|
||||||
|
|
||||||
While implementing all the features that mentioned above, the API has to handle multiple simultaneous requests.
|
|
||||||
Ideally, those requests should be handled as fast as possible.
|
|
||||||
|
|
||||||
The API should be implemented such that it can be easily expandable and maintainable, so that future improvements can happen.
|
|
||||||
|
|
||||||
The API should be consistent and easy to use, information on how to use the API should also be available to possible users.
|
|
||||||
|
|
||||||
The API should be structured as a REST JSON API, per the requirements.
|
|
||||||
The API should only accept inputs via the URL parameters of GET requests or via JSON on POST requests.
|
|
||||||
Binary formats can also be used to handle file upload and downloads, as transferring files via JSON extremely inefficient.
|
|
||||||
|
|
||||||
\subsection{Generation of Models}
|
|
||||||
The service should use any means available to generate models, such means can be:
|
|
||||||
\begin{multicols}{2}
|
|
||||||
\begin{itemize}
|
|
||||||
\item Templating.
|
|
||||||
\item Transfer Learning.
|
|
||||||
\item Database Search.
|
|
||||||
\item Pretrained Models with classification heads.
|
|
||||||
\end{itemize}
|
|
||||||
\end{multicols}
|
|
||||||
|
|
||||||
|
|
||||||
\subsection{Models Training}
|
|
||||||
% The Training process follows % TODO have a flow diagram
|
|
||||||
|
|
||||||
Model Training should be independent of image classification. A model training should not affect any current classification. The system could use multiple ways to achieve this, such as:
|
|
||||||
\begin{multicols}{2} % TODO think of more ways
|
|
||||||
\begin{itemize}
|
|
||||||
\item Separating the training to different machines.
|
|
||||||
\item Control the number of resources that training machine can utilize
|
|
||||||
\item Control the time when the shared training and inference machine can be used for training.
|
|
||||||
\item Allow users to have their own ``Runners'' where the training tasks can happen.
|
|
||||||
\end{itemize}
|
|
||||||
\end{multicols}
|
|
||||||
|
|
||||||
|
|
||||||
\subsection{Conclusion}
|
|
||||||
This section introduced multiple possible designs options for a service, that intends to achieve automated image classification, can follow to implement a robust system.
|
|
||||||
|
|
||||||
The next section will be discussing how the system was implemented and which of the possible design options were chosen when implementing the system.
|
|
||||||
|
|
||||||
\pagebreak
|
|
||||||
@ -1,73 +0,0 @@
|
|||||||
\section{Introduction} \label{sec:introduction}
|
|
||||||
This section will introduce the project: background, motives, aims, goals, and success criteria.
|
|
||||||
The section will end with this report structure.
|
|
||||||
|
|
||||||
\subsection{Project Background}
|
|
||||||
There are many automated tasks that being done manually.
|
|
||||||
If those tasks can be done automatically, a lot of productivity could be gained from as human doing those tasks can do tasks that only humans can.
|
|
||||||
|
|
||||||
This project aims to provide a software platform, where users with no experience in machine learning, data analysis could create machine learning models to process their data.
|
|
||||||
In this project, the platform will be scoped to image classification.
|
|
||||||
As an easy-to-use platform needs to be able to handle: image uploads, processing, and verification; model creation, management, and expansion; and image classification.
|
|
||||||
|
|
||||||
% This report will do a brief analysis of current image classification systems, followed by an overview of the design of the system, and implementation details. The report will finish with analysis of legal, ethical and societal issues, and evaluation of results, and objectives.
|
|
||||||
|
|
||||||
\subsection{Project Motivations}
|
|
||||||
|
|
||||||
Currently, there are many classification tasks that are being done manually.
|
|
||||||
Thousands of man-hours are used to classify images, this task can be automated.
|
|
||||||
There are a few easy-to-use image classification systems that require low to no knowledge of image classification.
|
|
||||||
This project aims to fill that role and provide a complete image classification service.
|
|
||||||
While still been user-friendly, where a user who has never done any kind of user classification still could get good results, by simply using this service.
|
|
||||||
|
|
||||||
\subsection{Project Aim}
|
|
||||||
The project aims to create an easy-to-use software platform, where users can create image classification models without having prior knowledge about machine learning.
|
|
||||||
The user should only need to upload the images and confirm, and the system should be able to perform all the steps necessary to create and manage the machine learning model.
|
|
||||||
|
|
||||||
\subsection{Project Objectives}
|
|
||||||
This project will have two different objectives.
|
|
||||||
Primary objectives are objectives that are required for the project to be considered a success.
|
|
||||||
Secondary objectives are objectives that are not required for the project to be considered a success, but they would provide a better experience for the user of the service.
|
|
||||||
|
|
||||||
This project's primary objectives are to design and implement:
|
|
||||||
\begin{itemize}
|
|
||||||
\item a system to upload images that will be assigned to a model
|
|
||||||
\item a system to automatically create and train models.
|
|
||||||
\item a platform where users can manage their models.
|
|
||||||
% \item a system to automatically expand and reduce models without fully retraining the models.
|
|
||||||
\item a system to automatically expand models without fully retraining the models.
|
|
||||||
\item an Application Programming Interface(API) that users can interact programmatically with the service.
|
|
||||||
\end{itemize}
|
|
||||||
|
|
||||||
This project's secondary objectives are to:
|
|
||||||
\begin{itemize}
|
|
||||||
% \item Create a system to automatically to merge modules to increase efficiency.
|
|
||||||
\item Create a system to distribute the load of training the model among multiple servers.
|
|
||||||
\end{itemize}
|
|
||||||
|
|
||||||
\subsection{Success Criteria}
|
|
||||||
As it was mentioned before, the project can be considered a success when the primary objectives have been completed.
|
|
||||||
|
|
||||||
Therefore, the success criteria of this project can be defined as:
|
|
||||||
|
|
||||||
\begin{itemize}
|
|
||||||
\item A user can upload images, train a model on those images, and evaluate images using a user interface.
|
|
||||||
\item A user can perform the same tasks, via the API service.
|
|
||||||
\end{itemize}
|
|
||||||
|
|
||||||
\subsection{Project Structure}
|
|
||||||
The report on the project shows the development and designs stages of the project. With each section addressing a part of the design and development process.
|
|
||||||
|
|
||||||
\renewcommand*{\arraystretch}{2}
|
|
||||||
\begin{longtable}{p{7cm} p{8cm}}
|
|
||||||
\hyperref[sec:introduction]{Introduction} & The introduction section will do a brief introduction of the project and its objectives. \\
|
|
||||||
\hyperref[sec:lit-tech-review]{Literature and Technical Review} & The Literature and Technical Review section will introduce some current existing projects that are similar to this one, and introduce some technologies that can be used to implement this project. \\
|
|
||||||
\hyperref[sec:sanr]{Service Analysis and Requirements} & This section will analyse the project requirements. The section will define design requirements that the service will need to implement to be able to achieve the goals that were set up. \\
|
|
||||||
\hyperref[sec:sd]{Service Design} & This section will discuss how a service could be designed that it matches the requirements of the service. \\
|
|
||||||
\hyperref[sec:sd]{Service Implementation} & Information on how the design of the system was turned into software is in this section. \\
|
|
||||||
\hyperref[sec:lsec]{Legal, Societal, Ethical, Professional Considerations} & This section will cover potential legal, societal, ethical and professional, issues that might arise from the service and how they are mitigated. \\
|
|
||||||
\hyperref[sec:se]{Service Evaluation} & In this section, the model will be tested and the results of the tests will be analysed. \\
|
|
||||||
\hyperref[sec:crpo]{Critical Review of Project Outcomes} & In this section, will compare the project goals with what was achieved. Then, according to the results, the project will either be deemed successful or not.
|
|
||||||
|
|
||||||
\end{longtable}
|
|
||||||
\pagebreak
|
|
||||||
@ -1,98 +0,0 @@
|
|||||||
\section{Literature and Technical Review} \label{sec:lit-tech-review}
|
|
||||||
This section reviews existing technologies in the market that do image classification. It also reviews current image classification technologies, which meet the requirements for the project. This review also analyses methods that are used to distribute the learning between various physical machines, and how to spread the load so minimum reloading of the models is required when running the model.
|
|
||||||
|
|
||||||
\subsection{Existing Classification Platforms}
|
|
||||||
There are currently some existing software as a service (SaaS) platforms that do provide similar services to the ones this will project will be providing.
|
|
||||||
|
|
||||||
%Amazon provides bespoque machine learning services that if were contacted would be able to provide image classification services. Amazon provides general machine learning services \cite{amazon-machine-learning}.
|
|
||||||
|
|
||||||
Amazon provides an image classification service called ``Rekognition'' \cite{amazon-rekognition}. This service provides multiple services from face recognition, celebrity recognition, object recognition and others. One of these services is called custom labels \cite{amazon-rekognition-custom-labels} that provides the most similar service, to the one this project is about. The custom labels service allows the users to provide custom datasets and labels and using AutoML the Rekognition service would generate a model that allows the users to classify images according to the generated model.
|
|
||||||
|
|
||||||
The models generated using Amazon's Rekognition do not provide ways to update the number of labels that were created, without generating a new project. This will involve retraining a large part of the model, which would involve large downtime between being able to add new classes. Training models also could take 30 minutes to 24 hours, \cite{amazon-rekognition-custom-labels-training}, which could result in up to 24 hours of lag between the need of creating a new label and being able to classify that label. A problem also arises when the uses need to add more than one label at the same time. For example, the user sees the need to create a new label and starts a new model training, but while the model is training a new label is also needed. The user now either stops the training of the new model and retrains a new one, or waits until the one currently running stops and trains a new one. If new classification classes are required with frequency, this might not be the best platform to choose.
|
|
||||||
|
|
||||||
%https://aws.amazon.com/machine-learning/ml-use-cases/
|
|
||||||
|
|
||||||
%https://aws.amazon.com/rekognition/image-features/
|
|
||||||
|
|
||||||
Similarly, Google also has ``Cloud Vision API'' \cite{google-vision-api} which provides similar services to Amazon's Rekognition. But Google's Vision API appears to be more targeted at videos than images, as indicated by their price sheet \cite{google-vision-price-sheet}. They have tag and product identifiers, where every image only has one tag or product. The product identifier system seams to work differently than the Amazon's Rekognition and worked based on K neighbouring giving the user similar products on not classification labels \cite{google-vision-product-recognizer-guide}.
|
|
||||||
|
|
||||||
This method is more effective at allowing users to add new types of products, but as it does not give defined classes as the output, the system does not give the target functionality that this project is aiming to achieve.
|
|
||||||
|
|
||||||
\subsection{Requirements of Image Classification Models}
|
|
||||||
|
|
||||||
The of the main objectives of this project are to be able to create models that can give a class given an image for any dataset. Which means that there will be no ``one solution fits all to the problem''. While the most complex way to solve a problem would most likely result in success, it might not be the most efficient way to achieve the results.
|
|
||||||
|
|
||||||
This section will analyse possible models that would obtain the best results. The models for this project have to be the most efficient as possible while resulting in the best accuracy as possible.
|
|
||||||
|
|
||||||
A classical example is the MNIST Dataset \cite{mnist}. Models for the classification of the MNIST dataset can be both simple or extremely complex and achieve different levels of complexity.
|
|
||||||
For example, in \cite{mist-high-accuracy} an accuracy $99.91\%$, by combining 3 Convolutional Neural Networks (CNNs), with different kernel sizes and by changing hyperparameters, augmenting the data, and in \cite{lecun-98} an accuracy of $95\%$ was achieved using a 2 layer neural network with 300 hidden nodes. Both these models achieve the accuracy that is required for this project, but \cite{mist-high-accuracy} are more computational intensive to run. When deciding when to choose what models they create, the system should choose to create the model that can achieve the required accuracy while taking the leas amount of effort to train.
|
|
||||||
|
|
||||||
% TODO fix the inglish in these sentance
|
|
||||||
The models for this system to work as indented should be as small as possible while obtaining the required accuracy required to achieve the task of classification of the classes.
|
|
||||||
|
|
||||||
As the service might need to handle many requests, it needs to be able to handle as many requests as possible. This would require that the models are easy to run, and smaller models are easier to run; therefore the system requires a balance between size and accuracy.
|
|
||||||
|
|
||||||
% TODO talk about storage
|
|
||||||
|
|
||||||
\subsection{Method of Image Classification Models}
|
|
||||||
|
|
||||||
There are all multiple ways of achieving image classification, the requirements of the system are that the system should return the class that an image that belongs to. Which means that we will be using supervised classification methods, as these are the ones that meet the requirements of the system.
|
|
||||||
|
|
||||||
% TODO find some papers to proff this
|
|
||||||
|
|
||||||
The system will use supervised models to classify images, using a combination of different types of models, using neural networks, convolution neural networks, deed neural networks and deep convolution neural networks.
|
|
||||||
|
|
||||||
These types were decided as they have had a large success in the past in other image classification challenges, for example in the ImageNet challenges \cite{imagenet}, which has ranked different models in classifying a 14 million images. The contest has been running since 2010 to 2017.
|
|
||||||
|
|
||||||
The models that participated in the contest tended to use more and more Deep convolution neural networks, out of the various models that were generated there are a few landmark models that were able to achieve high accuracies, including AlexNet \cite{krizhevsky2012imagenet}, ResNet-152 \cite{resnet-152}, EfficientNet \cite{efficientnet}.
|
|
||||||
% TODO find vgg to cite
|
|
||||||
|
|
||||||
These models can be used in two ways in the system, they can be used to generate the models via transfer learning and by using the model structure as a basis to generate a complete new model.
|
|
||||||
|
|
||||||
\subsection{Well-known models}
|
|
||||||
% TODO compare the models
|
|
||||||
|
|
||||||
This section will compare the different models that did well in the image net challenge.
|
|
||||||
|
|
||||||
AlexNet \cite{krizhevsky2012imagenet} is a deep convolution neural network that participated in the ImageNet ILSVRC-2010 contest, it achieved a top-1 error rate of $37.5\%$, and a top-5 error rate of $37.5\%$. A variant of this model participated in the ImageNet LSVRC-2012 contest and achieved a top-5 error rate of $15.3\%$. The architecture of AlexNet consists of 5 convolution layers that are run separately followed by 3 dense layers, some layers are followed by Max pooling. The training the that was done using multiple GPUs, one GPU would run the part of each layer, and some layers are connected between GPUs. The model during training also contained data argumentation techniques such as label preserving data augmentation and dropout.
|
|
||||||
While using AlexNet would probably yield desired results, it would complicate the other parts of the service. As a platform as a service, the system needs to manage the number of resources available, and requiring to use 2 GPUs to train a model would limit the number of resources available to the system by 2-fold.
|
|
||||||
% TODO talk more about this
|
|
||||||
|
|
||||||
ResNet \cite{resnet} is a deep convolution neural network that participated in the ImageNet ILSVRC-2015 contest, it achieved a top-1 error rate of $21.43\%$ and a top-5 error rate of $5.71\%$. ResNet was created to solve a problem, the problem of degradation of training accuracy when using deeper models. Close to the release of the ResNet paper, there was evidence that deeper networks result in higher accuracy results, \cite{going-deeper-with-convolutions, very-deep-convolution-networks-for-large-scale-image-recognition}. but the increasing the depth of the network resulted in training accuracy degradation.
|
|
||||||
% This needs some work in terms of gramar
|
|
||||||
ResNet works by creating shortcuts between sets of layers, the shortcuts allow residual values from previous layers to be used on the upper layers. The hypothesis being that it is easier to optimize the residual mappings than the linear mappings.
|
|
||||||
The results proved that the using the residual values improved training of the model, as the results of the challenge prove.
|
|
||||||
It's important to note that using residual networks tends to give better results, the more layers the model has. While this could have a negative impact on performance, the number of parameters per layer does not grow that steeply in ResNet when comparing it with other architectures as it uses other optimizations such as $1x1$ kernel sizes, which are more space efficient. Even with these optimizations, it can still achieve incredible results. Which might make it a good contender to be used in the service as one of the predefined models to use to try to create the machine learning models.
|
|
||||||
|
|
||||||
% MobileNet
|
|
||||||
|
|
||||||
% EfficientNet
|
|
||||||
EfficientNet \cite{efficient-net} is a deep convolution neural network that was able to achieve $84.3\%$ top-1 accuracy while ``$8.4x$ smaller and $6.1x$ faster on inference than the best existing ConvNet''. EfficientNets \footnote{the family of models that use the thecniques that described in \cite{efficient-net}} are models that instead of the of just increasing the depth or the width of the model, we increase all the parameters at the same time by a constant value. By not scaling only depth, EfficientNets can acquire more information about the images, specially the image size is considered.
|
|
||||||
To test their results, the EfficientNet team created a baseline model which as a building block used the mobile inverted bottleneck MBConv \cite{inverted-bottleneck-mobilenet}. The baseline model was then scaled using the compound method, which resulted in better top-1 and top-5 accuracy.
|
|
||||||
While EfficientNets are smaller than their non-EfficientNet counterparts, they are more computational intensive, a ResNet-50 scaled using the EfficientNet compound scaling method is $3\%$ more computational intensive than a ResNet-50 scaled using only depth while improving the top-1 accuracy by $0.7\%$.
|
|
||||||
And as the model will be trained and run multiple times decreasing the computational cost might be a better overall target for sustainability then being able to offer higher accuracies.
|
|
||||||
Even though scaling using the EfficientNet compound method might not yield the best results using some EfficientNets what were optimized by the team to would be optimal, for example, EfficientNet-B1 is both small and efficient while still obtaining $79.1\%$ top-1 accuracy in ImageNet, and realistically the datasets that this system will process will be smaller and more scope specific than ImageNet.
|
|
||||||
|
|
||||||
|
|
||||||
% \subsection{Efficiency of transfer learning}
|
|
||||||
|
|
||||||
% \subsection{Creation Models}
|
|
||||||
% The models that I will be creating will be Convolutional Neural Network(CNN) \cite{lecun1989handwritten,fukushima1980neocognitron}.
|
|
||||||
% The system will be creating two types of models that cannot be expanded and models that can be expanded. For the models that can be expanded, see the section about expandable models.
|
|
||||||
% The models that cannot be expanded will use a simple convolution blocks, with a similar structure as the AlexNet \cite{krizhevsky2012imagenet} ones, as the basis for the model. The size of the model will be controlled by the size of the input image, where bigger images will generate more deep and complex models.
|
|
||||||
% The models will be created using TensorFlow \cite{tensorflow2015-whitepaper} and Keras \cite{chollet2015keras}. These theologies are chosen since they are both robust and used in industry.
|
|
||||||
|
|
||||||
% \subsection{Expandable Models}
|
|
||||||
% The current most used approach for expanding a CNN model is to retrain the model. This is done by, recreating an entire new model that does the new task, using the older model as a base for the new model \cite{amazon-rekognition}, or using a pretrained model as a base and training the last few layers.
|
|
||||||
|
|
||||||
% There are also unsupervised learning methods that do not have a fixed number of classes. While this method would work as an expandable model method, it would not work for the purpose of this project. This project requires that the model has a specific set of labels which does not work with unsupervised learning which has unlabelled data. Some technics that are used for unsupervised learning might be useful in the process of creating expandable models.
|
|
||||||
|
|
||||||
|
|
||||||
\subsection{Conclusion}
|
|
||||||
The technical review of current systems reveals that there are current systems that exist that can perform image classification tasks, but they are not friendly in ways to easily expand currently existing models.
|
|
||||||
|
|
||||||
The current methods that exist for image classification seem to have reached a classification accuracy and efficiency that make a project like this feasible.
|
|
||||||
|
|
||||||
% TODO talk about web serving thechnlogies
|
|
||||||
|
|
||||||
\pagebreak
|
|
||||||
File diff suppressed because it is too large
Load Diff
156
report/sanr.tex
156
report/sanr.tex
@ -1,156 +0,0 @@
|
|||||||
\section{Service Analysis and Requirements} \label{sec:sanr}
|
|
||||||
Understanding the project that is being built is a critical step in the software deployment process.
|
|
||||||
This section will discuss what are the requirements that a service needs to implement for the project to be considered a success.
|
|
||||||
|
|
||||||
As a software as a service project, there are some required parts that the project needs to have:
|
|
||||||
\begin{itemize}
|
|
||||||
\item A way for the user to interact with the system
|
|
||||||
\item A way for programs to interact with the system
|
|
||||||
\item Management of images
|
|
||||||
\item Management of models
|
|
||||||
\item Management of compute resources
|
|
||||||
\end{itemize}
|
|
||||||
|
|
||||||
\subsection{Service Structure}
|
|
||||||
The service has to be structured so that users can interact with in two ways.
|
|
||||||
|
|
||||||
The first way is for the user to directly interface with the system using a user interface.
|
|
||||||
This interface does not have any strict form requirements, it could either be a desktop application, web application or even a command line application.
|
|
||||||
The main objective of this interface is for the user to quickly understand what the system is doing with their data, and if they can use the model they created to evaluate images.
|
|
||||||
|
|
||||||
The second way for the user to interface with the system needs to be an API.
|
|
||||||
This is required as it would not make sense for users to be able to quickly generate image classification models if they still had to evaluate all the images manually.
|
|
||||||
Therefore, there needs to be away for the user product to connect with the system, the API provides exactly that.
|
|
||||||
|
|
||||||
The system should also be structured in a way that allows easy scalability.
|
|
||||||
So that it can handle many requests at the same time.
|
|
||||||
The system should be able to scale, this could be achieved in many ways.
|
|
||||||
One way is by allowing the service to act as a cluster, where the same application is running multiple times and a load balancer, balances the load between the systems.
|
|
||||||
Another way is for the service to behave as a distributed system, where the services are split into smaller modules and those modules can be replicated.
|
|
||||||
Independently of how the system scales, it requires the ability to handle the fact that the data that the system uses not be available everywhere.
|
|
||||||
|
|
||||||
As a machine learning solution, the service requires the necessary computational power to handle the training of the models.
|
|
||||||
This means that the system needs to be structured in an away that it can decouple the training process from the main process.
|
|
||||||
Which guarantees that the compute requirements for training the model do not affect the main server running.
|
|
||||||
Ideally, the service should be able to divide the tasks from tasks that would require the GPU, and tasks that would require the CPU.
|
|
||||||
|
|
||||||
\subsection{Resources}
|
|
||||||
As a machine learning image classification service, the service has to manage various types of resources.
|
|
||||||
|
|
||||||
\subsubsection{Compute Resources}
|
|
||||||
As mentioned before, the service needs to be able to manage its compute resources.
|
|
||||||
This is required because, for example, if the system starts training a new model and that training uses all the GPU resources, it would impact the ability of the service to be able to evaluate images for other users.
|
|
||||||
As this example demonstrated, the system needs to keep track of the amount of GPU power available, so it can manage the actions it has to take accordingly.
|
|
||||||
|
|
||||||
Therefore, for optimal functionality, the service requires the management of various compute resources.
|
|
||||||
|
|
||||||
There should be a separation of the different kinds of compute power.
|
|
||||||
The two types of compute power are: CPU and GPU.
|
|
||||||
The CPU is needed to handle the multiple requests that the API might answer at the same time.
|
|
||||||
And the GPU resources are required to train models and evaluate the images.
|
|
||||||
|
|
||||||
As a result, the service needs a system to distribute these compute tasks.
|
|
||||||
The tasks have to be distributed between the application that is running the API and the various other places where that compute can happen.
|
|
||||||
|
|
||||||
An ideal system would distribute the tasks intelligently, to allow the maximization of resources.
|
|
||||||
An example of this would be running image classification, on the same model, on the same place twice, this would allow the model to stay in memory and not need to be reloaded again from disk.
|
|
||||||
These kinds of optimizations would help the system to be more efficient and less wasteful.
|
|
||||||
|
|
||||||
Another way to reduce the load that the system goes through is to allow users to add their own compute power to the system.
|
|
||||||
That compute power would only use images and models that are owned by the user.
|
|
||||||
While allowing the compute power to run any image or model in the system would allow for an even more scalable system, it would be an incredible violation of privacy and security.
|
|
||||||
As it allows outsiders access to possible sensitive information.
|
|
||||||
Which makes the idea of a complete distributed network of user provided compute power not viable.
|
|
||||||
|
|
||||||
\subsubsection{Storage}
|
|
||||||
Another resource that it has to handle is storage.
|
|
||||||
As the service accepts user uploaded images, the service has to monitor how much storage those images take.
|
|
||||||
The service will need systems to handle when the user uploaded images take too much space.
|
|
||||||
There are many ways of handling this, such as allowing the user to store their images, compacting the images, deleting images that the system might no longer need, or allowing dynamic storage services such as Object Buckets.
|
|
||||||
|
|
||||||
If there is not enough space to store all the images from all the models, and the service needs to delete images.
|
|
||||||
There should be a system that removes the images in a manner that causes the less harm.
|
|
||||||
An example of this would be deleting images in a way that keeps the dataset balanced.
|
|
||||||
|
|
||||||
If there is a discrepancy of where compute and storage happen, the system needs to be able to handle that.
|
|
||||||
This can be accomplished in various methods.
|
|
||||||
The most aggressive one is not allowing to compute resources to access data that is far away.
|
|
||||||
The less aggressive and smarter way is to allow the system to move data to the optimal place.
|
|
||||||
|
|
||||||
\subsection{User interface}
|
|
||||||
A service such as this requires a way for the users to quickly get an understating of how their data is being used and how they can perform the actions they want.
|
|
||||||
|
|
||||||
As previously mentions, this application can take multiple forms, from web apps, to command line applications.
|
|
||||||
As long as the application is easy to use, and allows the user to perform the required tasks:
|
|
||||||
\begin{itemize}
|
|
||||||
\item{Configure model}
|
|
||||||
\item{Upload images}
|
|
||||||
\item{Manage images}
|
|
||||||
\item{Request model training}
|
|
||||||
\item{Request image evaluation}
|
|
||||||
\item{Configure access}
|
|
||||||
% \item{See API usage}
|
|
||||||
%TODO write more
|
|
||||||
\end{itemize}
|
|
||||||
|
|
||||||
The way that the application communicates with the service should be done, via the API.
|
|
||||||
If there was a requirement to physical access the computer that the service is running on, it would defeat the purpose of this project.
|
|
||||||
Therefore, being able to control the service via the API makes the most reasonable sense.
|
|
||||||
A second system could be developed that allows the application to control the service, but that would be terribly inefficient.
|
|
||||||
Allowing the application to control the system via the API, also improves the API, as the API now gets more features.
|
|
||||||
|
|
||||||
The application should also allow administrators of the service to control the resources that are available to the system, to see if there is any requirement to add more resources.
|
|
||||||
|
|
||||||
\subsection{API} \label{sec:anal-api}
|
|
||||||
As a software as a service platform, most of the requests made to the service would be made via the API, not the user interface.
|
|
||||||
This is the case because the users that would need this service would set up the model using the web interface and then do the image classifications requests via the API.
|
|
||||||
|
|
||||||
While there are no hard requirements for the user interface, that is not the case for the API.
|
|
||||||
The API must be implemented as an HTTPS REST API, this is because the most of the APIs that currently exist online are HTTPS REST APIs \cite{json-api-usage-stats}.
|
|
||||||
If the service wants to be easy to use, it needs to be implemented in away such that it has the lowest barrier to entry.
|
|
||||||
Making the type of the API a requirement would guarantee that the application would be the most compatible with other systems that already exist.
|
|
||||||
|
|
||||||
The API needs to be able to do all the tasks that the application can do.
|
|
||||||
|
|
||||||
The API also requires authentication.
|
|
||||||
This is needed to prevent users from:
|
|
||||||
\begin{itemize}
|
|
||||||
\item{Modifying systems settings}
|
|
||||||
\item{Accessing other users' data}
|
|
||||||
\end{itemize}
|
|
||||||
The API must implement authentication methods to prevent those kinds of actions from happening.
|
|
||||||
|
|
||||||
\subsection{Data Management}
|
|
||||||
The service will store a large amount of user data.
|
|
||||||
This includes: user information, user images, user models.
|
|
||||||
|
|
||||||
\subsubsection*{User Information}
|
|
||||||
There are no hard requirements on how the user information needs to be stored, as long as it is done securely.
|
|
||||||
User information includes personal identifiable information such as username and email, and secret information such as passwords, and access tokens.
|
|
||||||
|
|
||||||
Future versions of the service could possible also store more sensitive information about the user, such, as payment information and addresses.
|
|
||||||
Such information is required if the user needs to be charged, but payment for the services provided is outside the scope of this project.
|
|
||||||
|
|
||||||
\subsubsection*{User Images}
|
|
||||||
|
|
||||||
Images are another kind of information that has to be stored.
|
|
||||||
As it was mentioned before, the system has to keep track of the images and the space they use.
|
|
||||||
The system should also guarantee that there is some level of security in accessing the images that were uploaded to the service.
|
|
||||||
|
|
||||||
\subsubsection*{Models}
|
|
||||||
The last kind of data that the service has to keep track of is model data.
|
|
||||||
Once the model is trained, it has to be saved on disk.
|
|
||||||
The service should implement a system that manages where the models are stored.
|
|
||||||
This is similar to the image situation, where the model should be as close as possible to the compute resource that is going to utilize it, even if this requires copying the model.
|
|
||||||
|
|
||||||
\subsection{Conclusion}
|
|
||||||
This section shows that there are requirements that need to be met for the system to work as indented. These requirements range from usability requirements, implementation details, to system-level resource management requirements.
|
|
||||||
|
|
||||||
The most important requirement is for the system to be easy to use by the user.
|
|
||||||
As if it's difficult to use, then the service already fails in one of its objectives.
|
|
||||||
|
|
||||||
The other requirements are significant as well, as without them, the quality of the service would be very degraded.
|
|
||||||
And even if the service was effortless to use, it is as bad as being difficult to use if it could not process the images quickly in a reasonable amount of time.
|
|
||||||
|
|
||||||
The next section will describe a design that matches a subset of the requirements.
|
|
||||||
\pagebreak
|
|
||||||
@ -1,70 +0,0 @@
|
|||||||
\usepackage[english]{babel} % English language/hyphenation
|
|
||||||
\usepackage{url}
|
|
||||||
\usepackage{tabularx}
|
|
||||||
\usepackage{pdfpages}
|
|
||||||
\usepackage{float}
|
|
||||||
\usepackage{longtable}
|
|
||||||
\usepackage{multicol}
|
|
||||||
|
|
||||||
\usepackage{graphicx}
|
|
||||||
\usepackage{svg}
|
|
||||||
\graphicspath{ {../images for report/} }
|
|
||||||
\usepackage[margin=2cm]{geometry}
|
|
||||||
|
|
||||||
\usepackage{datetime}
|
|
||||||
\newdateformat{monthyeardate}{\monthname[\THEMONTH], \THEYEAR}
|
|
||||||
|
|
||||||
\usepackage{hyperref}
|
|
||||||
\hypersetup{
|
|
||||||
colorlinks,
|
|
||||||
citecolor=black,
|
|
||||||
filecolor=black,
|
|
||||||
linkcolor=black,
|
|
||||||
urlcolor=black
|
|
||||||
}
|
|
||||||
|
|
||||||
\usepackage{cleveref}
|
|
||||||
|
|
||||||
%%% Custom headers/footers (fancyhdr package)
|
|
||||||
\usepackage{fancyhdr}
|
|
||||||
\pagestyle{fancyplain}
|
|
||||||
|
|
||||||
% \fancyhead{}
|
|
||||||
|
|
||||||
\fancypagestyle{my_empty}{%
|
|
||||||
\fancyhf{}
|
|
||||||
\renewcommand{\headrulewidth}{0pt}
|
|
||||||
\renewcommand{\footrulewidth}{0pt}
|
|
||||||
}
|
|
||||||
|
|
||||||
\fancypagestyle{simple}{%
|
|
||||||
\fancyhf{}
|
|
||||||
\renewcommand{\headrulewidth}{0pt}
|
|
||||||
\renewcommand{\footrulewidth}{0pt}
|
|
||||||
\fancyfoot[L]{} % Empty
|
|
||||||
\fancyfoot[C]{\thepage} % Pagenmbering
|
|
||||||
\fancyfoot[R]{} % Empty
|
|
||||||
}
|
|
||||||
|
|
||||||
\fancypagestyle{full}{%
|
|
||||||
\fancyhf{}
|
|
||||||
\renewcommand{\headrulewidth}{0.5pt}
|
|
||||||
\renewcommand{\footrulewidth}{0pt}
|
|
||||||
\fancyfoot[L]{} % Empty
|
|
||||||
\fancyfoot[C]{\thepage} % Pagenmbering
|
|
||||||
\fancyfoot[R]{} % Empty
|
|
||||||
|
|
||||||
\fancyhead[RO,LE]{Andre Henriques}
|
|
||||||
}
|
|
||||||
|
|
||||||
\renewcommand{\headrulewidth}{0pt} % Remove header underlines
|
|
||||||
\renewcommand{\footrulewidth}{0pt} % Remove footer underlines
|
|
||||||
\setlength{\headheight}{13.6pt}
|
|
||||||
|
|
||||||
\newcommand*\NewPage{\newpage\null\thispagestyle{empty}\newpage}
|
|
||||||
|
|
||||||
% numeric
|
|
||||||
\usepackage[bibstyle=ieee, citestyle=numeric, sorting=none,backend=biber]{biblatex}
|
|
||||||
\addbibresource{../main.bib}
|
|
||||||
|
|
||||||
\raggedbottom
|
|
||||||
@ -1,60 +0,0 @@
|
|||||||
\pagenumbering{gobble}
|
|
||||||
|
|
||||||
\maketitle
|
|
||||||
\pagestyle{my_empty}
|
|
||||||
|
|
||||||
\begin{center}
|
|
||||||
\includegraphics[height=0.5\textheight]{uni_surrey}
|
|
||||||
\end{center}
|
|
||||||
|
|
||||||
\begin{center}
|
|
||||||
\monthyeardate\today
|
|
||||||
\end{center}
|
|
||||||
|
|
||||||
\NewPage
|
|
||||||
\pagenumbering{arabic}
|
|
||||||
|
|
||||||
\pagestyle{simple}
|
|
||||||
|
|
||||||
\begin{center}
|
|
||||||
\vspace*{\fill}
|
|
||||||
\section*{Declaration of Originality}
|
|
||||||
I confirm that the submitted work is my own work and that I have clearly identified and fully
|
|
||||||
acknowledged all material that is entitled to be attributed to others (whether published or
|
|
||||||
unpublished) using the referencing system set out in the programme handbook. I agree that the
|
|
||||||
University may submit my work to means of checking this, such as the plagiarism detection service
|
|
||||||
Turnitin® UK. I confirm that I understand that assessed work that has been shown to have been
|
|
||||||
plagiarised will be penalised.
|
|
||||||
\vspace*{\fill}
|
|
||||||
\end{center}
|
|
||||||
\NewPage
|
|
||||||
|
|
||||||
\begin{center}
|
|
||||||
\vspace*{\fill}
|
|
||||||
\section*{Acknowledgements}
|
|
||||||
I would like to take this opportunity to thank my supervisor, Rizwan Asghar that helped me with this project from the start of the until the end.
|
|
||||||
His help with the report was incredibly useful.
|
|
||||||
|
|
||||||
I would like to thank my family and friends for their support and encouragement from the beginning.
|
|
||||||
\vspace*{\fill}
|
|
||||||
\end{center}
|
|
||||||
\NewPage
|
|
||||||
|
|
||||||
\begin{center}
|
|
||||||
\vspace*{\fill}
|
|
||||||
\section*{Abstract}
|
|
||||||
Currently there is a log of man-hours used performing tasks that can be done by automated systems.
|
|
||||||
If a user, without any knowledge of image classification, can create an image classification model with ease, it would allow those man-hours to be used for more productive.
|
|
||||||
|
|
||||||
This project aims to develop a classification platform where users can create image classification models with as few clicks as possible.
|
|
||||||
The project will create multiple systems that allow: model creation, model raining, and model inference.
|
|
||||||
|
|
||||||
This report will guide the reader through the ideas and designs that were implemented.
|
|
||||||
\vspace*{\fill}
|
|
||||||
\end{center}
|
|
||||||
\NewPage
|
|
||||||
|
|
||||||
\tableofcontents
|
|
||||||
\newpage
|
|
||||||
|
|
||||||
\pagestyle{full}
|
|
||||||
Loading…
Reference in New Issue
Block a user