You are reading the article How To Reference Embedded Docker Resource Files Using File Path Url? updated in November 2023 on the website Minhminhbmm.com. We hope that the information we have shared is helpful to you. If you find the content interesting and meaningful, please share it with your friends and continue to follow and support us for the latest updates. Suggested December 2023 How To Reference Embedded Docker Resource Files Using File Path Url?
IntroductionEmbedded Docker resource files are files included in a Docker image rather than stored on the host file system or an external network location. These files can be useful for including configuration files, scripts, or other resources that are needed by the applications or processes running in the Docker container.
You can reference embedded Docker resource files in several different ways, including using file path URLs. This article explains what file path URLs are and how to use them to reference embedded Docker resource files. We will also provide tips and examples to help you use file path URLs effectively in your Dockerfiles and Docker Compose files.
Using file path URLs to reference embedded Docker resource filesFile path URLs use the file:// protocol to reference a file on the local file system. In the context of Docker, file path URLs can reference embedded Docker resource files by specifying the path to the file within the Docker image.
Here is an example of using a file path URL to reference an embedded Docker resource file in a Dockerfile −
FROM ubuntu COPY resources /app/resources CMD ["cat", "file:///app/resources/config.txt"]In this example, the resources directory is copied into the /app/resources directory in the Docker image. The CMD directive then runs the cat command to display the contents of the chúng tôi file, which is located at file:///app/resources/config.txt within the Docker image.
ExampleHere is an example of using a file path URL to reference an embedded Docker resource file in a Docker Compose file −
In this example, the resources directory on host is mounted as a volume in the /app/resources directory in the Docker container. The command field in the service definition then runs the cat command to display the contents of the chúng tôi file, which is located at file:///app/resources/config.txt within the Docker container.
Tips for using file path URLs to reference embedded Docker resource filesHere are some tips to help you use file path URLs effectively when referencing embedded Docker resource files −
Make sure to use the correct path to the embedded resource file. The path should start with file:// and should be the path to the file within the Docker image or container.
Be aware that file path URLs can only be used to reference files embedded in the Docker image or mounted as volumes in the Docker container. They cannot be used to reference files on the host file system or external network locations.
If you are having trouble using file path URLs, try using absolute paths rather than relative paths. Absolute paths are more reliable and less prone to error, especially when using file path URLs.
Alternative methods for referencing embedded Docker resource filesHere are some alternative methods for referencing embedded Docker resource files −
Using environment variables: You can use environment variables to specify the path to an embedded Docker resource file. For example, you can set the RESOURCE_FILE environment variable to the path of the resource file in the Dockerfile and then reference the environment variable in the CMD directive:
FROM ubuntu ENV RESOURCE_FILE /app/resources/config.txt COPY resources /app/resources CMD ["cat", "$RESOURCE_FILE"]
Using ADD or COPY directives: You can use the ADD or COPY directives in a Dockerfile to copy the resource file from the Docker image to a location on the host file system. This command allows you to reference the resource file using a regular file path on the host file system.
FROM ubuntu ADD resources/config.txt /app/config.txt CMD ["cat", "/app/config.txt"]
Using volume mounts: If you are using Docker Compose, you can mount a host directory as a volume in the Docker container and then reference the resource file using a file path within the volume.
version: "3" services: app: build: . command: ["cat", "/app/resources/config.txt"] volumes: - type: bind source: ./resources target: /app/resources ConclusionThis article explained what file path URLs are and how to use them to reference embedded Docker resource files. We provided examples and tips to help you use file path URLs effectively in your Dockerfiles and Docker Compose files. We also discussed alternative methods for referencing embedded Docker resource files, including environment variables, ADD or COPY directives, and volume mounts.
Using file path URLs is a convenient way to reference embedded Docker resource files, especially if you want to avoid copying the resource files to the host file system or using environment variables. However, choosing the right method for referencing embedded Docker resource files is important based on your specific needs and requirements.
You're reading How To Reference Embedded Docker Resource Files Using File Path Url?
How To Get The File Name From The File Path In Python?
In this article, we will learn a python program to get the file name from the file Path.
Methods UsedThe following are the various methods to accomplish this task −
Using the OS module functions
Using Pathlib Module
Using Regular expressions(regex module)
Method 1: Using the OS Module Functions Get the File Name From the File Path using the split() FunctionThe split() function splits a string into a list. We can define the separator; the default separator is any whitespace.
Algorithm (Steps)Following are the Algorithms/steps to be followed to perform the desired task. −
Use the import keyword to import the os module.
Create a variable to store the input file path.
Use the split() function to split the file path into a list of words based on the ‘/’ separator.
Get the last element from the list using negative indexing(indexing from the end i.e -1, -2….).
print the resultant filename.
ExampleThe following program returns the filename from the given file path using the split() function −
# importing os module import os # input file path inputFilepath = 'C:/Users/cirus/Desktop/tutorialsPoint.pdf' # Printing the given file path print("The given file path is:",inputFilepath) # splitting the file path into a list of words based on '/' and # getting the last element using negative indexing print("The File Name is:n",os.path.basename(inputFilepath).split('/')[-1]) OutputOn execution, the above program will generate the following output −
The given file path is: C:/Users/cirus/Desktop/tutorialsPoint.pdf The File Name is: tutorialsPoint.pdf Get the File Name From the File Path using the os.path.basenameUsing the built-in Python function os.path.basename(), one may determine the base name in the specified path. The base name of the pathname path is returned by the function path.basename(), which takes a path argument.
ExampleThe following program returns the filename from the given file path using os.path.basename() function −
# importing os module import os # input path of the file inputFilepath = 'C:/Users/cirus/Desktop/tutorialsPoint.pdf' # Printing the given input path print("Give Input Path is:",inputFilepath) # getting the last component(main file name )of the input file path print("The File Name is:n",os.path.basename(inputFilepath)) OutputOn execution, the above program will generate the following output −
Give Input Path is: C:/Users/cirus/Desktop/tutorialsPoint.pdf The File Name is: tutorialsPoint.pdf Get the File Name From the File Path using the os.splitext()If we only need the file name without an extension or only extensions, this method will produce a file and its extension. Here, the os module’s splitext function comes into play.
The os.splitext() method will return a tuple of strings that includes the filename and the content, which we can access using indexing.
ExampleThe following program returns the filename from the given file path using os.splitext() function −
# importing os module import os inputFilepath = 'C:/Users/cirus/Desktop/tutorialsPoint.pdf' # Printing the given input path print("Give Input Path is:",inputFilepath) # getting the file name from the file path fileName = os.path.basename(inputFilepath) # splitting the file using the splittext() function full_file = os.path.splitext(fileName) # printing the tuple of a string containing file name and extension separately print(full_file) # Concatenating file name with file extension using string concatenation print("The File Name is:n",full_file[0] + full_file[1]) Output Give Input Path is: C:/Users/cirus/Desktop/tutorialsPoint.pdf ('tutorialsPoint', '.pdf') The File Name is: tutorialsPoint.pdf Method 2: Using Pathlib ModuleSeveral classes that describe file system paths with semantics appropriate for numerous operating systems are available in the Python Pathlib module. This module is one of the fundamental Python utility modules.
If we want an extension with the file, we can utilize name attributes, even if the stem is one of the utility attributes that enables extracts of the filename from the link without extension.
ExampleThe following program returns the filename from the given file path using Path() function and stem attribute of Pathlib Module −
# importing Path from pathlib module from pathlib import Path # input file path inputFilepath = 'C:/Users/cirus/Desktop/tutorialsPoint.pdf' # getting the filename from the file path # here the stem attribute extracts the file name from filepath print('File Name is:',Path(inputFilepath).stem) # here the name attribute returns full name(along with extension) # of the input file print("The File Name Along with Extension is:",Path(inputFilepath).name) Output File Name is: tutorialsPoint The File Name Along with the Extension is: tutorialsPoint.pdf Method 3: Using Regular expressions(regex module)In order to match the file name with the particular pattern, we can use a regular expression.
pattern - [w]+?(?=.)The above pattern is divided into 3 patterns −
[w] − matches the words inside the set
+? − matches the string if appears just once before the ? keyword
(?=) − matches any character without a newline and Keep in mind that to stop at.
regex re.search() methodThe Python regex re.search() method searches the entire target string for occurrences of the regex pattern and returns the appropriate Match Object instance where the match was found. Only the first match to the pattern in the target string is returned by re.search().
ExampleThe following program returns the filename from the given file path using the Regular expressions (regex) −
# importing re(regex) module import re # input file path inputFilepath = 'C:/Users/cirus/Desktop/tutorialsPoint.pdf' # regex pattern to extract the file name regex_pattern = '[w-]+?(?=.)' # searching/matching the pattern of the input file path result = re.search(regex_pattern, inputFilepath) # printing the match name print("The File Name is:",result.group()) Output The File Name is: tutorialsPoint ConclusionIn this article, we learned how to use the three different methods to get the file name from the given file path. We learned how to modify the provided file path using the built-in features of the OS module to meet our needs.
How To Batch Change File Extensions For Windows Files
Recently, I came across a problem where I had to change the file extension on a bunch of Windows files from no extension to .JPG. Someone sent me a large number of files, but for some strange reason, they had no file extension, even though they were images! Once I added the .JPG extension to the end of the file, I was able to open the images.
There are many other reasons why you may need to rename file extensions, so I’ll show you how you can change them one at a time if you have a small number of files or batch rename file extensions if you have lots of files.
Table of Contents
If you have a file with no extension, Windows will probably assign a generic icon to it. When you try to open it, the Open With dialog will pop up where you’ll get a list of programs to choose from.
Obviously, since there is no file extension for the file, Windows is not able to determine which program should open it. If you want to change the file extension for the file, you first need to be able to see file extensions in Windows. You can do this by going to My Computer and then going to Tools and Folder Options.
Go to the View tab and scroll down till you see the option “Hide extensions for known file types“. Go ahead and uncheck it.
Move to the end of the name and type in .XXX or whatever the type is for the file you want it to be, i.e. .TXT for text files, .XLS for Excel files, etc. Obviously, the file has to have originally been from that program that you are changing the file extension too. In my case, the text file was from Excel, so I added that extension back.
How to Batch Rename File Extensions
So how do you rename multiple file extensions at once? If you have a directory full of files that need to be changed from .PNG to .JPG, it’s going to take forever to do this using the method above. Luckily, there are a couple of different ways we can change the file extension for multiple files at once.
Command LineFor those of you who are familiar with the command line, there is a very easy command you can use to perform very simple file renaming tasks. There are two commands that you can use at the command prompt: ren and rename. Both of them do the exact same thing, so it’s just a preference as to which one you use.
If you have a folder with a bunch of JPGs inside and you want to change the file extension to PNG, for example, you would type in the following command:
ren *.jpg *.png
As you can see above, the command changed all the files in that directory with a JPG file extension to ones with a PNG file extension. You can get more information on how to use the ren command from Microsoft’s website.
Bulk Rename Utility
I mentioned this tool even though it’s overkill for our purpose because it’s a really great utility and one that some might be interested in learning if they want to rename thousands of photos with names likes DSC00x, IMG00x, etc.
Advanced RenamerNow go ahead and type in the new extension you would like into the box at the top and then change the Apply to box at the bottom to Extension instead of Name.
Again, I’m giving the simplest example with these programs, but you can create far more complex renaming schemes if you like. If you really don’t care about all the extra functionality, then check out the last program that does nothing but change the file extension.
Bulk Extension ChangerIf you want simple, Bulk Extension Changer is the program for you. All you have to do is three things: first, pick the folder where the files are located, then set the current and replacement extension and then just press the Go button.
The only option is if you want to include sub-directories or not. In the 2nd step, you can add multiple replacement rules in case your folder has files of many different types and you want to check several at once.
How To Copy Files To A New Directory Using Python?
The act of copying files to a new directory is a basic skill that every developer in Python should invariably possess. Whether it is the process of backing up data, organizing files or creating a new project, there are powerful tools provided by Python to make file copying an easy task. In this extensive article, you will learn the process of copying files to a new directory using Python. Detailed explanations and practical code examples, will help you gain the knowledge and confidence to handle file copying tasks successfully.
Understanding File CopyingBefore we begin with the examples in this article, let us take a moment to make sense of the concept of file copying. File copying means duplicating the contents of one file and storing it in or transferring it to a different location. This process enables us to take backups, move files around among directories, or even create several versions of the same file.
Copying a Single FileLet us consider a simple code example where we learn how to copy a single file to a new directory in Python.
ExampleIn this example, the shutil module is imported; it makes provisions for high-level file operations. We assign the path of the file we want to copy to the source_file variable. Next, we declare the destination_directory variable, which indicates the directory where we want to keep the copied file. Finally, the shutil.copy() function is made use of to perform the file copy operation.
import shutil # Specify the source file path source_file = '/path/to/your/source/file.txt' # Specify the destination directory path destination_directory = '/path/to/your/destination/directory/' # Copy the file to the destination directory shutil.copy(source_file, destination_directory) Copying Multiple FilesOftentimes, we may want to copy several files to a new directory at a go. The following code example shows you how to achieve this in Python.
ExampleIn this code example, we import the glob module together with the shutil module. We declare the source_directory variable to show the path to the directory containing the files we wish to copy or duplicate. Similarly, we declare the destination_directory variable as the target directory where we wish to keep the copied files. By using glob.glob() function with the pattern source_directory + ‘/*’, we fetch a list of all files in the source directory. We then loop over each file and copy it to the destination directory making use of shutil.copy().
import shutil import glob # Specify the source directory path source_directory = '/path/to/your/source/directory/' # Specify the destination directory path destination_directory = '/path/to/your/destination/directory/' # Get a list of all files in the source directory files = glob.glob(source_directory + '/*') # Copy each file to the destination directory for file in files: shutil.copy(file, destination_directory) Copying Files with RenamingIn certain situations, we may want to copy files to a new directory after renaming the files. The code example given below shows how to accomplish this task in Python.
ExampleIn this code, we assign the source_file variable with the path of the file we wish to copy. We then declare the destination_file variable with the path and the new desired name of the copied file which is to be renamed.
By utilizing shutil.copy() function and giving the destination file path, we copy the file to the new directory while renaming it at the same accordingly.
import shutil # Specify the source file path source_file = '/path/to/your/source/file.txt' # Specify the destination file path with the desired name destination_file = '/path/to/your/destination/new_file.txt' # Copy the file to the destination directory with renaming shutil.copy(source_file, destination_file) OutputFor some particular source file, the output can be
/content/sample_data/newactive_config Handling Errors and ExceptionsIt’s essential and important to handle possible errors and exceptions that may take place or happen during file copying operations. The following example shows you how to integrate error handling while you are copying the file.
ExampleHere, in this snippet, we use a try-except block to tackle possible exceptions that may occur while the file copying process is underway. We make an attempt to copy the file making use of shutil.copy() within the try block. If a FileNotFoundError occurs, it implies that the source file does not exist. If an IsADirectoryError occurs, it indicates that the destination path points to a directory instead of a file. A PermissionError may occur if there are inadequate permissions to copy the file. At last, the except Exception block acts as a catch-all to handle any other unforeseen errors that may occur. By incorporating and integrating error handling, we can skillfully handle various scenarios during file copying.
import shutil # Specify the source file path source_file = '/path/to/your/source/file.txt' # Specify the destination directory path destination_directory = '/path/to/your/destination/directory/' try: # Copy the file to the destination directory shutil.copy(source_file, destination_directory) print("File copied successfully!") except FileNotFoundError: print("Source file not found.") except IsADirectoryError: print("Destination is a directory.") except PermissionError: print("Permission denied.") except Exception as e: print("An error occurred:", str(e)) OutputFor some particular source file, the output can be
File copied successfully!In this comprehensive and extensive article, we navigated and explored the process of copying files to a new directory using Python. We considered code examples for copying a single file, copying several files, copying files along with renaming them and handling errors and exceptions that may arise. With these strategies at your disposal as your resource, you now can manage file-copying tasks with confidence in your Python projects. Keep in mind to consider the source file path, the destination directory, and any particular requirements such as renaming files. By doing file copying in Python like an expert, you can efficiently organize, backup, and manipulate files with ease, making you realize your coding prowess.
How To Extract Tabular Data From Doc Files Using Python?
This article was published as a part of the Data Science Blogathon
IntroductionData is present everywhere. Any action we perform generates some or the other form of data. But this data might not be present in a structured form. A beginner starting with the data field is often trained for datasets in standard formats like CSV, TSV, or some text files. CSV files are the most preferred files as they can be loaded into a pandas dataframe and manipulated more easily. The text files can be loaded using naive Python file handling modules.
But in the real world, any type of document can have the data needed for analysis. While I was applying for an internship position in a company, my assignment was to draw analysis out of the data present in the Doc file. In this article, I will explain the ETL process for a Doc file, the difference between Doc and Docx extensions, conversion of Doc to Docx, and at the end, I will show you how I created some interactive plots from that data.
Image by Author, Made in Canva
Difference Between Doc and DocxWhile dealing with doc files, you will come across these two extensions: ‘.doc’ and ‘.docx’. Both the extensions are used for Microsoft word documents that can be created using Microsoft Word or any other word processing tool. The difference lies in the fact that till word 2007, the “doc” extension was used extensively.
After this version, Microsoft introduced a new extension, “Docx”, which is a Microsft Word Open XML Format Document. This extension allowed files to be smaller, easy to store, and less corrupted. It also opened doors to online tools like Google Sheets which can easily manage these Docx files.
Conversion of Doc to Docx in PythonToday, all the files are by default created with the extension Docx but there are still many old files with Doc extension. A Docx file is a better solution to store and share data but we can’t neglect the data stored in Doc files. It might be of great value. Therefore, to retrieve data from Doc files, we need to convert the Doc file to Docx format. Depending on the platform, Windows or Linux, we have different ways for this conversion.
For WindowsManually, for a word file to be saved as Docx, you simply need to save the file with the extension “.docx”
We will perform this task using Python. Window’s Component Object Model (COM) allows Windows applications to be controlled by other applications. pywin32 is the Python wrapper module that can interact with this COM and automate any windows application using Python. Therefore, the implementation code goes like this:
from win32com import client as wc w = wc.Dispatch('Word.Application') doc = w.Documents.Open("file_name.doc") doc.SaveAs("file_name.docx", 16)Breakdown of the code:
First, we are importing the client from the win32com package which is preinstalled module during Python installation.
Next, we are creating a Dispatch object for the Word Application.
Then, we are opening this document and saving it with the Docx extension.
For Linux
We can directly use LibreOffice in-build converter:
lowriter --convert-to docx testdoc.doc Reading Docx files in PythonPython has a module for reading and manipulating Docx files. It’s called “python-docx”. Here, all the essential functions have been already implemented. You can install this module via pip:
pip install python-docxI won’t go into detail about how a Docx document is structured but on an abstract level, it has 3 parts: Run, paragraph, and Document objects. For this tutorial, we will be dealing with paragraph and Document objects. Before moving to the actual code implementation, let us see the data will be extracting:
Data in new Docx file
The new Docx file contains the glucose level of a patient after several intervals. Each data row has an Id, Timestamp, type, and glucose level reading. To maintain anonymity, I have blurred out the Patient’s name. Procedure to extract this data:
1. Import the module
import docx2. Create a Docx file document object and pass the path to the Docx file.
Text = docx.Document('file_name.docx')3. Create an empty data dictionary
data = {}4. Create a paragraph object out of the document object. This object can access all the paragraphs of the document
paragraphs = Text.paragraphs5. Now, we will iterate over all the paragraphs, access the text, and save them into a data dictionary
for i in range(2, len(Text.paragraphs)): data[i] = tuple(Text.paragraphs[i].text.split('t'))Here I had to split the text at “t” as if you look at one of the rows, it had the tab separator.
6. Access the values of the dictionary
data_values = list(data.values())Now, these values are transformed as a list and we can pass them into a pandas dataframe. According to my use case, I had to follow some additional steps such as dropping unnecessary columns and timestamp conversion. Here is the final pandas dataframe I got from the initial Doc file:
There are a lot of things that can be done using the python-docx module. Apart from loading the file, one can create a Docx file using this module. You can add headings, paragraphs, make text bold, italics, add images, tables, and much more! Here is the link to the full documentation of the module.
Bonus Step: Plot using PlotlyThe main aim of this article was to show you how to extract tabular data from a doc file into a pandas dataframe. Let’s complete the ELT cycle and transform this data into beautiful visualizations using the Plotly library! If you don’t know, Plotly is an amazing visualization library that helps in creating interactive plots.
These plots don’t require much effort as most of the things can be customized. There are many articles on Analytics Vidhya describing the usage of this library. For my use case, here is the configuration for the plot:
import plotly.graph_objects as go fig = go.Figure() fig.add_trace(go.Scatter(x=doc_data.index, y=doc_data['Historic Glucose (mg/dL)'].rolling(5).mean(), mode='lines', marker=dict( size=20, line_width=2, colorscale='Rainbow', showscale=True, ), name = 'Historic Glucose (mg/dL)' )) fig.update_layout(xaxis_tickangle=-45, font=dict(size=15), yaxis={'visible': True}, xaxis_title='Dates', yaxis_title='Glucose', template='plotly_dark', title='Glucose Level Over Time' ) fig.update_layout(hovermode="x")Image by Author
ConclusionIn this article, I explained what are doc files, the difference between Doc and Docx file extensions, conversion of Doc files into Docx files, loading and manipulation of Docx files, and finally how to load this tabular data into a pandas dataframe.
If you want to read/explore every article of mine, then head over to my master article list which gets updated every time I publish a new article on any platform!
For any doubts, queries, or potential opportunities, you can reach out to me via:
1. Linkedin — in/kaustubh-gupta/
2. Twitter — @Kaustubh1828
3. GitHub — kaustubhgupta
4. Medium — @kaustubhgupta1828
The media shown in this article on Interactive Dashboard using Bokeh are not owned by Analytics Vidhya and are used at the Author’s discretion.
Related
How To Create Your Own Question And Answering Api(Flask+Docker +Bert) Using Haystack Framework
Introduction
Note from the author: In this article, we will learn how to create your own Question and Answering(QA) API using python, flask, and haystack framework with docker. The haystack framework will provide the complete QA features which are highly scalable and customizable. In this article Medium Rules, the text will be used as the target document and fine-tuning the model as well.
Basic Knowledge Required: Elasticsearch & Docker
This article contains the working code which can be directly build using docker.
Content of the article
DocumentStores set up — Installation of Elasticsearch
API Code Explanation
Build the Flask API in docker
Load data from the API
Demonstration of the API
Fine-tuning a model
Code tuning (Trained/Pre-Trained)
Summarize Result
1. DocumentStores set upRun the below command for the installation of Elasticsearch
verify the installation by browsing the below URL. If install successfully it will display as in below:
2. API Code ExplanationHaystack framework has main 3 basic components DocumentStore, Retriever, and Reader which we have to select based on our requirements.
DocumentStore: As recommended ElasticsearchDocumentStore will be used in this article. It comes preloaded with features like full-text queries, BM25 retrieval, and vector storage for text embeddings. Documents should be chunked into smaller units (e.g. paragraphs) before indexing to make the results returned by the Retriever more granular and accurate.
Retrievers: The answer needs to be display based on similarities on the embedded. So DensePassageRetriever is the powerful alternative to score the similarity of texts.
Readers: Farm Reader going to be used in this article if you wish Transformer readers can be used. For better accuracy “distilbert-base-uncased-distilled-squad” model going to be used by the readers.
The app structure as shown below:
Code Explanation for main.pyIn this article, we are going to use the ElasticSearch document. The application configuration is declared as in below:
Let’s implement endpoints for uploading the PDF document. The pdf document will be uploaded in the ElasticSearch with the provided index name.
Let’s implement endpoints for querying and will respond back with relevant (n)answers from the ElasticSearch document. The index needs to provide during the search query.
chúng tôifarm-haystack
Docker FileThe Question and Answering API will be available @port 8777.
3. Build the Flask API in dockerLet’s run the below command to build the docker image:
docker build -t qna:v1 .
Build and run the flask API in docker container using the below command:
docker run — name qna_app -d -p 8777:8777 xxxxxxxxx
Note: xxxxxxxxx is the image id
Confirm the docker container using the below command:
docker ps
Its will show all the process as in below:
4. Load data from the APILet’s prepare a pdf document from Medium Rules which suppose to be upload in an elastic search document. Once you prepare the pdf document lets to upload using the API as in the below snapshot:
Verify the upload document as in the below snapshot:
5. Demonstration of the APILet’s ask a question using the qna_pretrain endpoints. Currently, we have used only the pre-train model “distilbert-base-uncased-distilled-squad” which is provided good accuracy. In the next article, I will demonstrate how to annotate and improve the model. For querying a question use the API as in the below snapshot:
6. Fine-tuning a modelLet’s try to improve the model and train ourselves as per our requirements. For any domain-specific training we have to train ourselves, the Haystack Annotation tool will be used for labeling our data. In this article, I will use the hosted version of the annotation tool from the haystack.
Now create an account with your personal email id and upload the txt document. Annotate your question and answers as in the below snapshot:
Now export all your labels in SQUAD format as “answers.json” and copy the file to the docker container with the below command:
Code explanation util-trainer.pyAfter the collection of the training labels, this python utility will help us to train and fine-tuning the model base on our requirements. Let’s proceed with the training with 10 epochs.
Use the below docker command to execute from the docker container:
To start the training run the command “python chúng tôi “
At the end of the training, the model will be saved in the train_model directory. Now training is successfully completed.
7. Code Tuning (Trained & Untrained) ModelLet’s modify the code to subscribed the trained and pre-trained model in the same endpoint. We are going to add the additional parameter “mode” to be classified the model.
Published the code to the docker contained with the below command:
docker copy chúng tôi qna_app:/usr/src/app/main.py
Now restart the container to reflect the changes in the API
docker restart qna_app
Now the API is ready to subscribe with the trained & pre-trained model.
8. Summarize ResultNow, let’s compare the two models (trained and pre-trained) and analyze the most probability answer. Use the below API as in the snapshot:
the trained model gives the perfect results as we provided the label in training data.
I hope we have learned to develop our own Question and Answering API by fine-tuning the model.
The same article is published on my medium channel
The complete source code is available here
The media shown in this article are not owned by Analytics Vidhya and is used at the Author’s discretion.
Related
Update the detailed information about How To Reference Embedded Docker Resource Files Using File Path Url? on the Minhminhbmm.com website. We hope the article's content will meet your needs, and we will regularly update the information to provide you with the fastest and most accurate information. Have a great day!