THE RED QUEEN (HR) V2.0
第二代紅皇后 (人力資源)
Know insights about how TKEG, as a small size consulting agency, utilizes Artificial Intelligence to automate its recruitment.
瞭解奕資 作為一家小型諮詢機構如何利用人工智慧實現招聘自動化。
An Easy-To-Implement Integration That Bends Into Your Organization
一個融入您組織的易實施集成
As easy as making a few API calls. The TKEG Red Queen automates every step in a recruiting process, from formatting application data to make the final decision on each applicant.
像做幾次應用程式介面傳呼一樣簡單。奕資紅皇后自動化所有會在招募階段發生的事。從整理申請信息，到做最後的抉擇。
We haven’t commercialized this system yet. However, we would like to share what we have developed so far. This system simply integrates many natural language machine learning systems and then runs them in your centralized platform.
我們尚未商業化這個系統，然而，我們希望與您分享我們已經建成的。這個系統簡單的集成一些自然語言機器學習習戎，然後在您中心化平台上運行。
RESEARCH ARTICLE
研究文獻
The Development of An AI Algorithm based on Sensitivity Hypothesis of Early-stage Filtering Based on Application Result Prediction by Assessing Opportunity Seeker's Written Responses and LinkedIn Profile Using AI Labeling Algorithm in an Automated Talent-recruit Workflow to Increase Labor Efficiency and Streamline Application Reviewing Process in a Developing Consulting Agency
發展中的諮詢機構通過在自動化人才招聘工作流中使用人工智慧標記算灋評估求職者的書面回復和領英檔案開發基於應用結果預測的早期篩選的人工智慧算灋以提高勞動效率並簡化申請審查流程之敏感性假設
06 2021
Director 指揮 ： KEITH YUNXI ZHU 朱耘希
Supervisor 監視 ： OLIVER T DALMI 達米.奧利佛
Co-Authors 作者 ： Yuhan Kang, Jingyang Wu, India Gloag
1
Background
背景
The Development of An AI Algorithm based on Sensitivity Hypothesis of Early-stage Filtering Based on Application Result Prediction by Assessing Opportunity Seeker's Written Responses and LinkedIn Profile Using AI Labeling Algorithm in an Automated Talent-recruit Workflow to Increase Labor Efficiency and Streamlines Application Reviewing Process in a Developing Consulting Agency
在自動化人才招聘工作流程中使用人工智能標籤算法評估求職者的書面回復和領英檔案，以提高勞動效率並簡化申請審查流程，從而開發基於敏感性假設的早期篩選的人工智能算法。
Online recruiting platforms are revolutionary; they streamline the application evaluation process and reduce time and labor costs. However, only a few can provide AI-driven application evaluation. This study showcases how TKEG applies AI and Machine learning to increase the efficiency of the application review process in any recruitments.
在線招聘平台是革命性的； 它們簡化了應用程序評估過程並減少了時間和勞動力成本。 但是，只有少數的平台可以提供人工智能驅動的申請評估。 本研究展示了奕資如何應用人工智能和機器學習來提高任何招聘中的申請審查流程的效率。
2
Problems With The Existing Online Recruiting Platforms
現有網上招聘平台的問題
Suppose companies continue using platforms such as LinkedIn and Glassdoor, which do not provide any AI-powered application evaluation service. Application reviewers typically spend significant amounts of time manually reading through resumes and written responses to identify key elements in each application. A recruiter spends on average 23 hours screening resumes for a single hire [2]. Indeed, manually screening resumes is still the most time-consuming part of recruiting.
假設公司繼續使用領英和 Glassdoor 等平台，這些平台不提供任何人工智能驅動的申請評估服務。 申請審核人員通常會花費大量時間手動閱讀簡歷和書面回复，以確定每個申請中的關鍵要素。 招聘人員平均要花費 23 小時來篩選單個招聘人員的簡歷[2]。 事實上，手動篩選簡歷仍然是招聘中最耗時的部分。
Existing AI-powered recruitment systems usually operate independently from popular recruiting platforms, give limited API/integration options, and do not support "apply with one click" feature that popular recruiting platforms have, making candidates harder to apply. Moreover, application reviewers typically have to work across multiple platforms, making data less centralized and more challenging to get a complete picture of each applicant. The decentralisation of information causes companies to spend around $590 million every year trying to integrate data from different platforms [3].
現有的人工智能招聘系統通常獨立於流行的招聘平台，提供有限的API/集成選項，並且不支持流行招聘平台具有的“一鍵申請”功能，使候選人更難申請。 此外，申請審查員通常必須跨多個平台工作，這使得數據不那麼集中，並且更難以全面了解每個申請人。信息的去中心化導致公司每年需要花費約 5.9 億美元試圖整合來自不同平台的數據 [3]。
3
Problems Small/Medium Enterprises Face
中小企業面臨的問題
Small to Medium Size companies usually have particular needs in talents. Therefore, they either have to do large-scale recruitment campaigns or outsourcing to talent agencies.
中小型公司通常對人才有特殊需求。 因此，他們要么進行大規模的招聘活動，要么外包給人才機構。
Small to Medium Enterprises usually either have no HR division or have a minimal budget on HRs. In fact, only 47% of small businesses have even one person in an HR role [4]. However, talent agencies usually charge one to three months of successful candidates' salary, and manually evaluating a high quantity of applicants is incredibly time-consuming and requires a big budget on the HRs. Therefore, small to medial size enterprises (SMEs) either have to give up obligations for recruiting or are unable to find the best possible candidate. According to Indeed, the #1 job site in the world, 81% of small business owners believe recruitment is more difficult for them compared to their large rivals [5].
中小型企業通常要么沒有人力資源部門，要么在人力資源方面的預算很少。 事實上，只有 47% 的小企業甚至有一個人擔任人力資源的角色 [4]。 然而，人才中介機構通常會收取一到三個月的成功候選人的工資，人工評估大量的申請人非常耗時，並且需要人力資源部的大量預算。 因此，中小型企業 (SME) 要么不得不放棄招聘義務，要么無法找到最佳候選人。 根據世界排名第一的招聘網站 Indeed，81% 的小企業主認為與大型競爭對手相比，他們的招聘難度更大 [5]。
4
What Did TKEG Do to Overcome This Common Difficulty
奕資如何解決這個常見的難題
Unlike the majority of Small/Medium Enterprises, the presence of TKEG and its charisma attracted a considerable number of people wanting to join this team. From March 2021 to June 2021, TKEG received 1,480 resumes, 775 written responses, and 335 completed video interview records for internship positions.
與大多數中小型企業不同，奕資的存在及其魅力吸引了相當多的人想要加入這個團隊。 2021年3月至2021年6月，奕資收到1480份簡歷、775份書面回復和335份實習崗位視頻面試記錄。
Since applicants started flooding into TKEG's candidate pool, we understood that our capacity could not process that many applications if we handled them manually, but as a growing startup, we did not want to give up a single chance of finding an ideal future partner to go on this journey with us. Thus a long-planned development of TKEG's automated recruitment system started.
由於申請人開始湧入奕資的候選人庫，我們明白如果我們手動處理，我們的能力無法處理那麼多申請，但作為一家成長中的初創公司，我們不想放棄尋找理想的未來合作夥伴的任何機會。 因此，奕資的自動化招聘系統的長期計劃開發開始了。
4.1
Redesign the Workflow
重新設計工作流水線
TKEG first redesigned the workflow of application evaluation, streamlined the process to reduce time and workforce costs. Considering assigning candidates to different HR specialists would result in low efficiency and ununified decision-making standards. Therefore, instead of assigning candidates to HRs and expect them to review the whole application, TKEG broke the application process into different stages and assigned each stage to be reviewed by different HR specialists for maximum efficiency, less bias, and most importantly, a consistency of stander, which benefited TKEG enormously in its later AI training process.
奕資首先重新設計了申請評估的工作流程，簡化了流程以減少時間和人力成本。 考慮將候選人分配給不同的人力資源專家會導致低下的效率和不統一的決策標準。 因此，奕資不是將候選人分配給人力資源專家並期望他們審查整個申請，而是將申請過程分成不同的階段，並將每個階段分配給不同的人力資源專家進行審查，以最大限度地提高效率，減少偏見，最重要的是，保持一致性。這使得奕資在後期的人工智能訓練過程中受益匪淺。
4.2
Train the Model
訓練模型
After redesigning the workflow, TKEG started uploading every manual evaluation for AI to practice Entity Extraction and Natural Language Processing in identifying key elements in each applicant's resume, written responses, online presence, and interview record to indicate an applicant's fitness and interest in the company and the strength we look for in successful applicants.
重新設計工作流程後，奕資開始將每個人工評估上傳給人工智能，以練習實體提取和自然語言處理，以確定每個申請人簡歷、書面回复、在線狀態和麵試記錄中的關鍵元素，以表明申請人對公司的適合度和興趣，以及是否具有我們在成功申請者中尋找的優勢。
{ "displayName": "MODEL_DISPLAY_NAME", "trainingTaskDefinition": "gs://google-cloud-aiplatform/schema/trainingjob/definition/automl_text_classification_1.0.0.yaml", "trainingTaskInputs": { "multiLabel": MULTI-LABEL } "modelToUpload": { "displayName": "MODEL_DISPLAY_NAME" }, "inputDataConfig": { "datasetId": "DATASET_ID" } }
Training an accurate model with a limited dataset is impossible. Therefore, TKEG gave up on regression evaluation, instead classified applications between "Above Average," "Average," and "Below Average," each with a respective confidence score, summing to 1 for each candidate. As we process more applicants and grow our data set, we seek to implement more advanced but data-intensive models to replace this classification process.
用有限的數據集訓練準確的模型是不可能的。 因此，奕資放棄了回歸評估，而是在“高於平均水平”、“平均水平”和“低於平均水平”之間對申請進行分類，每個應用程序都有各自的置信度得分，每個候選者的總和為 1。 隨著我們處理更多申請人並擴大我們的數據集，我們尋求實施更先進但數據密集型的模型來取代這種分類過程。
In the recent upgrade of THE RED QUEEN (HR), a total of 408 samples resulted in average precision of 77.9%. However, since it did not pass our 94% threshold, we decided to only take the confidence score of the label “poor” as a reliable reference in our later decision-making process.
在近期的紅皇后(人力資源)升級中，408個樣本導致最後的77.9%準確率。由於並沒有通過我們94%的門檻，我們決定只適用標籤“差”的自信指數作為日後我們的決定議程中的參考。
4.3
Achieved 100% Automation in Final Decision Making
在最後決策方面實現了絕對自動化
After TKEG received 500 applications, we proposed a sensitivity hypothesis to retrodict an algorithm to calculate each criterion's assessment score and determine the chance of an application being admitted by a human admission selector. This algorithm helped TKEG filter out a significant amount of applicants (around 75%) to be manually processed in the final stage of review.
在奕資收到 500 份申請後，我們提出了一個敏感性假設，以追溯一種算法來計算每個標準的評估分數並確定申請被人工錄取選擇者錄取的機會。該算法幫助奕資過濾掉大量申請人（約 75%），以便在審查的最後階段進行人工處理。
IF({是否奇怪}="奇怪的人", IF({AI Score (weight)}<2,-1, IF({AI Score (weight)}<2.5,-0.8, IF({AI Score (weight)}<3,-0.6, IF({AI Score (weight)}<3.5,0.2, IF({AI Score (weight)}<4,0.4, IF({AI Score (weight)}<4.5,0.6, IF({AI Score (weight)}<5,0.8,1))))))), 0) IF({AI Score}<2.5,"WTF ❓❓", IF({AI Score (weight)}<2,"WTF ❓❓", IF({AI Score}<3.5,"🈲️🈲️🈲️🈲️", IF({AI Score (weight)}<2.5,"🈲️🈲️🈲️🈲️", IF({AI Score}>4.74999,"😱😱😱😱", IF({AI Score}>4.249999,"❤️❤️❤️❤️", IF(AND({Questions Result}="Okay",{Company-Awareness}="Okay")=TRUE,"🉑️🈶️🉑️🈚️", IF({AI Score}>3.9999999,"✅✅✅✅", IF({AI Score (weight)}>3.749999,"✅✅✅✅","🉑️🈶️🉑️🈚️")))))))))
5
The Problem With TKEG's Existing Recruitment Process
奕資現有招聘環節的問題
Although TKEG was already able to filter out most unqualified applicants in the final stage with its existing AI decision making system, the human involvement required in the video interview evaluation process was still considerable since all applicants were invited to the video interview.
儘管奕資已經能夠通過其現有的人工智能決策系統在最後階段過濾掉大多數不合格的申請人，但由於所有申請人都被邀請參加視頻面試，因此視頻面試評估過程中所需的人工參與仍然相當多。
TKEG desired to develop another model to filter out applications that have no chance of admission in the early stage of application before sending them an invitation to interview. This is especially important as up to 88% of resumes received for a role are considered unqualified [2]. By doing this, it would allow the company to focus on human evaluation of only applications with a possibility of successful admission.
奕資希望開發另一種模型，在發送面試邀請之前，過濾掉在申請早期沒有機會被錄取的申請。 這一點尤其重要，因為高達 88% 的職位簡歷被認為是不合格的 [2]。 通過這樣做，這將使公司能夠專注於對有可能成功錄取的申請進行人工評估。
6
The Development of THE RED QUEEN (HR) automated AI workflow
紅皇后（人力資源）人工智能自動化工作組之研發
The filtering criteria we settled through experimentation. TKEG began by analyzing all the key elements in an applicant's early-stage application and then studied the relationship between an application's early-stage assessment score, which includes manual evaluation of the applicant's LinkedIn profile and AI evaluation on written application applicant's final assessment score.
我們通過實驗確定的過濾標準。奕資首先分析了申請人早期申請中的所有關鍵要素，然後研究了申請早期評估分數之間的關係，包括對申請人領英個人資料的人工評估和對書面申請申請人最終評估分數的人工智能評估。
During TKEG's recruitment evaluation, we straightly deny candidates with a final AI score (weight) of less than 2.5 or with a final AI score of less than 3.5.
在奕資的招聘評估中，我們直接拒絕最終人工智能分數（權重）低於2.5或最終人工智能分數低於3.5的候選人。
After experimenting with parameters, we found that none of the applications with a 'Below Average' confidence score greater than 0.4 (in red) were accepted into the program.
在對參數進行試驗後，我們發現“低於平均水平”置信度得分大於 0.4（紅色）的應用程序均未被該程序接受。
Additionally, candidates with an "Above Average" confidence score of less than 0.15 who also have a "Below Average" online presence were never accepted into the program. (in orange)
此外，“高於平均水平”置信度低於 0.15 且在線狀態“低於平均水平”的候選人從未被接受進入該計劃。 （橙色）
In the end, these parameters serve as a highly effective model to predict what candidates will perform well in the entire recruitment process without gathering any information beyond just their answers to the questions on the application.
最後，這些參數作為一個高效的模型來預測哪些候選人將在整個招聘過程中表現出色，而無需收集除了他們對申請中問題的回答之外的任何其他信息。
7
The Outcome
結果
Of the 205 total data points, our methodology filtered the bottom 18 candidates into the red category, 47 into the orange category, and 140 into the green category. In doing this, we were able to remove 9% of applicants from consideration due to being red. Additionally, 23% of applicants were effectively removed from consideration by being in the orange category. By leaving 68% of candidates in our pool, we were able to keep more than enough candidates in the running to avoid accidentally filtering out a top candidate, but also significantly reduce the workload and resource needs of our HR department, who now would have to look at 32% fewer applicants.
在 205 個總數據點中，我們的方法將後 18 個候選者過濾到紅色類別中，將 47 個過濾到橙色類別中，將 140 個過濾到綠色類別中。 在此過程中，我們能夠將 9% 的申請人因紅色而排除在考慮範圍之外。 此外，23% 的申請人因屬於橙色類別而被有效地排除在考慮之外。 通過將 68% 的候選人留在我們的儲備庫中，我們能夠保留足夠多的候選人，以避免意外過濾掉最優秀的候選人，同時也顯著減少了我們人力資源部門的工作量和資源需求，他們現在必須評估的申請者減少了 32%。
While there remain many exciting improvements and expansions to this technology, making for an even more automated system, we at TKEG have already found our existing model to be a vital tool in streamlining our recruiting operations and increasing the number of applications we can accurately and effectively process, without having to grow our staff. Indeed, recruiting automation is likely to play a bigger role in the future, as three quarters of recruiters say technology is playing a bigger role in their hiring process this year [1].
雖然這項技術仍有許多令人興奮的改進和擴展，從而使系統更加自動化，但我們已經發現我們現有的模型是簡化我們的招聘操作和增加我們可以準確有效地評估的申請數量的重要工具，同時無需增加我們的員工。事實上，招聘自動化在未來可能會發揮更大的作用，因為四分之三的招聘人員表示，今年科技在他們的招聘過程中發揮著更大的作用 [1]。
References:
參考文獻：
Recruiting Daily (2020). 'How Recruiting Automation Can Improve Small Business Hiring', How%20Recruiting%20Automation%20Can%20Improve%20Small%20Business%20Hiring%20%26gt%3B%20Sourcing%20and%20Recruiting%20News
Ideal (2020). '3 Ways Recruitment Automation Will Change the Talent Acquistion Process', 3%20Ways%20Recruitment%20Automation%20Will%20Change%20Recruiting%20Forever%20%20Ideal
Retorio (2020). 'Video Interview for Recruitment? Make Sure You Get The Right API',retorio.com/...
Market Finance (2016). 'How to Manage HR in a Small Business', blog.marketfinance.com/...
Indeed (2018). 'Report: Addressing Hiring Challenges for Small Businesses', Report%3A%20Addressing%20Hiring%20Challenges%20for%20Small%20Businesses%20-%20Indeed%20Blog%20%20UK