Judge jobs in san francisco USA
112 judge jobs found in san francisco: showing 1 - 50
Software Engineer Ii - Security Review Automation
Company: Uber |
agents, model routing, or model-as-judge techniques to improve the accuracy, coverage, and confidence of automated securityLocation: San Francisco, CA, USA
| Salary: unspecified | Date posted: 04 Oct 2026
Security Audit & Controls, Security Grc
Company: Anthropic |
fluency to read a runbook, a configuration, or a pipeline definition and judge whether it enforces what the written controlLocation: San Francisco, CA - New York City, NY, USA
| Salary: unspecified | Date posted: 02 Oct 2026
Principal Machine Learning Engineer, Secureai
Company: Okta |
frameworks. Familiarity with evaluation datasets, synthetic data generation, or LLM-as-a-judge methods. Education:BachelorLocation: San Francisco, CA, USA
| Salary: unspecified | Date posted: 01 Oct 2026
Propulsion Systems Engineer - Expert
Company: Mercor |
, or adversarial. Evaluate AI model responses against policy standards. Judge the correctness of each response. Write referenceLocation: San Francisco, CA, USA
| Salary: US$65 - 75 per hour | Date posted: 30 Sep 2026
Industrial Hygienist - Safety Expert
Company: Mercor |
, or adversarial. Evaluate AI model responses against a defined policy standard. Judge correctness of handling. Write referenceLocation: San Francisco, CA, USA
| Salary: US$65 - 75 per hour | Date posted: 30 Sep 2026
Radiological Safety Expert - Redteaming
Company: Mercor |
with technical reasoning for correct responses. Collaborate with experts to ensure models correctly judge misuse potentialLocation: San Francisco, CA, USA
| Salary: US$65 - 75 per hour | Date posted: 30 Sep 2026
Litigation Practice Team Assistant
Company: Orrick Herrington & Sutcliffe LLP |
Rules of Court and Judge's Standing Orders. Expertise with PACER and other electronic filing systems in state and federalLocation: San Francisco, CA, USA
| Salary: US$84000 - 112000 per year | Date posted: 30 Sep 2026
Research Engineer, Training And Environment Infrastructure
Company: Product Pulse |
Francisco. Nice to have Experience with LLM-as-judge systems, automated graders, or rubric design for model outputsLocation: San Francisco, CA, USA
| Salary: unspecified | Date posted: 29 Sep 2026
Synthetic Chemist - Fully Remote
Company: Mercor |
. Evaluate AI model responses against a defined policy standard. Judge the correctness of each response. Write reference answersLocation: San Francisco, CA, USA
| Salary: US$65 - 75 per hour | Date posted: 28 Sep 2026
Policy Research, Strategic Advisory
Company: Anthropic |
to scope a research question, work with economists, academics and other experts to answer it, and judge the qualityLocation: San Francisco, CA, USA
| Salary: unspecified | Date posted: 24 Sep 2026
Toxicological Chemist - Fully Remote
Company: Mercor |
. Evaluate model responses against a defined policy standard and judge correctness. Write reference answers with technicalLocation: San Francisco, CA, USA
| Salary: US$65 - 75 per hour | Date posted: 22 Sep 2026
Explosives Specialist - Incident Analyst
Company: Mercor |
. Label them as benign, dual-use, or adversarial. Evaluate model responses against a defined policy standard. Judge whetherLocation: San Francisco, CA, USA
| Salary: US$65 - 75 per hour | Date posted: 22 Sep 2026
Intern, Ai Agent Evaluations
Company: G2 |
that can be tested. Write the tasks, test data, and scoring rules for those steps. Review the scores our AI judge and humanLocation: San Francisco, CA, USA
| Salary: unspecified | Date posted: 21 Sep 2026
Safeguards Access Operations Specialist (hyrbrid, Contract)
Company: Employer.com |
: Cybersecurity background (security ops, threat intel, vuln research) strong enough to judge whether a security use case is credibleLocation: San Francisco, CA, USA
| Salary: unspecified | Date posted: 21 Sep 2026
Process Safety Engineer - Fully Remote
Company: Mercor |
, or adversarial. Evaluate AI model responses against a defined policy standard. Judge correctness of handling. Write referenceLocation: San Francisco, CA, USA
| Salary: US$65 - 75 per hour | Date posted: 20 Sep 2026
Founding Senior Manager, Forward Deployed Engineering
Company: Afresh |
in view at the same time and judge where the two meet, without needing to be the deepest expert in any single layerLocation: San Francisco, CA, USA
| Salary: unspecified | Date posted: 19 Sep 2026
Principal Engineer, Agentic Applications
Company: Gap |
gates (LLM-as-judge, golden datasets) rather than subjective opinion Architect Agent-Ready Data. Shape the data strategy... with at least one major cloud AI stack (GCP/Vertex preferred). You've built or owned LLM-as-judge pipelines, golden datasets, or comparableLocation: San Francisco, CA, USA
| Salary: unspecified | Date posted: 19 Sep 2026
Commercial Blasting Engineer - Expert
Company: Mercor |
. Evaluate model responses against a defined policy standard. Judge the correctness of each response. Write reference answersLocation: San Francisco, CA, USA
| Salary: US$65 - 75 per hour | Date posted: 18 Sep 2026
Fp&a Expert - Financial Forecaster
Company: Mercor |
AI-agent rollouts in Financial Forecaster. Judge agent answers against the actual model and documents. Identify failure modesLocation: San Francisco, CA, USA
| Salary: unspecified | Date posted: 18 Sep 2026
Supplier Security & Assurance, Security Grc
Company: Anthropic |
user entity controls, and judge what the evidence does and does not prove Preferred qualifications Experience assessingLocation: San Francisco, CA, USA
| Salary: unspecified | Date posted: 17 Sep 2026
Server
Company: Arch Amenities Group |
, recognize, judge, observe, inspect, estimate, assess X Carry weight, lift -- Move, transport, position, put, install, removeLocation: San Francisco, CA, USA
| Salary: US$22 per hour | Date posted: 17 Sep 2026
Group Fitness Instructor
Company: Arch Amenities Group |
, express oneself, exchange information X See -- Detect, determine, perceive, identify, recognize, judge, observe, inspectLocation: San Francisco, CA, USA
| Salary: unspecified | Date posted: 17 Sep 2026
Personal Trainer
Company: Arch Amenities Group |
with, discern, convey, express oneself, exchange information X See -- Detect, determine, perceive, identify, recognize, judgeLocation: San Francisco, CA, USA
| Salary: unspecified | Date posted: 16 Sep 2026
Public Safety Specialist - Fully Remote
Company: Mercor |
. Label them as benign, dual-use, or adversarial. Evaluate responses against a defined policy standard. Judge whetherLocation: San Francisco, CA, USA
| Salary: US$65 - 75 per hour | Date posted: 16 Sep 2026
Nuclear Security Expert - Red Team
Company: Mercor |
for correct responses. Collaborate on dual-use scenarios, ensuring models accurately judge misuse potential. Work independentlyLocation: San Francisco, CA, USA
| Salary: US$65 - 75 per hour | Date posted: 16 Sep 2026
Synthetic Chemist - Fully Remote
Company: Mercor |
. Evaluate AI model responses against a defined policy standard. Judge the correctness of each response. Write reference answersLocation: San Francisco, CA, USA
| Salary: US$65 - 75 per hour | Date posted: 16 Sep 2026
Toxicological Chemist - Fully Remote
Company: Mercor |
. Evaluate model responses against a defined policy standard and judge correctness. Write reference answers with technicalLocation: San Francisco, CA, USA
| Salary: US$65 - 75 per hour | Date posted: 16 Sep 2026
Nuclear Safeguards Specialist - Fully Remote
Company: Mercor |
for correct responses. Collaborate on dual-use scenarios, ensuring models accurately judge misuse potential. Work independentlyLocation: San Francisco, CA, USA
| Salary: US$65 - 75 per hour | Date posted: 16 Sep 2026
Effects Engineering Specialist - Fully Remote
Company: Mercor |
, or adversarial. Evaluate AI model responses against policy standards. Judge the correctness of each response. Write referenceLocation: San Francisco, CA, USA
| Salary: US$65 - 75 per hour | Date posted: 16 Sep 2026
Forensic Materials Expert - Fully Remote
Company: Mercor |
. Label them as benign, dual-use, or adversarial. Evaluate model responses against a defined policy standard. Judge whetherLocation: San Francisco, CA, USA
| Salary: US$65 - 75 per hour | Date posted: 16 Sep 2026
Propulsion Chemist - Fully Remote
Company: Mercor |
. Classify them as benign, dual-use, or adversarial. Evaluate model responses against a defined policy standard. Judge theLocation: San Francisco, CA, USA
| Salary: US$65 - 75 per hour | Date posted: 16 Sep 2026
Process Safety Engineer - Fully Remote
Company: Mercor |
, or adversarial. Evaluate AI model responses against a defined policy standard. Judge correctness of handling. Write referenceLocation: San Francisco, CA, USA
| Salary: US$65 - 75 per hour | Date posted: 16 Sep 2026
Blasting Specialist - Fully Remote
Company: Mercor |
. Evaluate model responses against a defined policy standard. Judge the correctness of each response. Write reference answersLocation: San Francisco, CA, USA
| Salary: US$65 - 75 per hour | Date posted: 16 Sep 2026
Machine Learning Engineer (evals And Voice Models)
Company: Aircall |
models, agent capability evals, benchmark design, LLM-as-judge systems, failure analysis, and the infrastructure that ties it... evaluation data, and calibrate LLM-as-judge systems against human raters to ensure automated evals stay trustworthy over timeLocation: San Francisco, CA, USA
| Salary: unspecified | Date posted: 16 Sep 2026
Forensic Chemist - Casework Specialist - Ai Trainer
Company: Mercor |
. Evaluate model responses against a defined policy standard and judge correctness. Write reference answers with technicalLocation: San Francisco, CA, USA
| Salary: US$65 - 75 per hour | Date posted: 15 Sep 2026
Energetic Materials Chemist - Specialist
Company: Mercor |
. Classify them as benign, dual-use, or adversarial. Evaluate model responses against a defined policy standard. Judge theLocation: San Francisco, CA, USA
| Salary: US$65 - 75 per hour | Date posted: 15 Sep 2026
Chemistry Specialist - Fully Remote
Company: Mercor |
. Judge the calibration of prompts, distinguishing between mechanism and procedure. Work independently and asynchronouslyLocation: San Francisco, CA, USA
| Salary: US$65 - 75 per hour | Date posted: 14 Sep 2026
Radiological Safety Expert - Redteaming
Company: Mercor |
with technical reasoning for correct responses. Collaborate with experts to ensure models correctly judge misuse potentialLocation: San Francisco, CA, USA
| Salary: US$65 - 75 per hour | Date posted: 14 Sep 2026
Energetic Materials Expert - Redteaming
Company: Mercor |
, dual-use, or adversarial. Evaluate model responses against a defined policy standard. Judge the correctnessLocation: San Francisco, CA, USA
| Salary: US$65 - 75 per hour | Date posted: 14 Sep 2026
Nuclear Safeguards Expert - Red Team
Company: Mercor |
, and adversarial. Evaluate AI model responses against a defined policy standard to judge correctness. Write reference answersLocation: San Francisco, CA, USA
| Salary: US$65 - 75 per hour | Date posted: 14 Sep 2026
Chemical Safety Expert - Redteaming
Company: Mercor |
. Judge the calibration of prompts, distinguishing between mechanism and procedure. Work independently and asynchronouslyLocation: San Francisco, CA, USA
| Salary: US$65 - 75 per hour | Date posted: 14 Sep 2026
Radiological Safety Expert - Redteaming - Ai Trainer
Company: Mercor |
with technical reasoning for correct responses. Collaborate with experts to ensure models correctly judge misuse potentialLocation: San Francisco, CA, USA
| Salary: US$65 - 75 per hour | Date posted: 14 Sep 2026
Sales Expert - Judge Calibration
Company: Mercor |
. Position: Study Domain Expert Judge Calibration Sales Type: Contract Compensation: $90/hour Location: Remote DurationLocation: San Francisco, CA, USA
| Salary: unspecified | Date posted: 12 Sep 2026
Senior Fullstack Engineer - Product Platform Engineering
Company: Pivotal Health |
without needing each to be familiar first Strong product instincts: you judge a system by whether it is useful and correctLocation: San Francisco, CA - Santa Monica, CA, USA
| Salary: US$19000 - 22000 per year | Date posted: 12 Sep 2026
Lead Cook
Company: Arch Amenities Group |
, perceive, identify, recognize, judge, observe, inspect, estimate, assess X Carry weight, lift -- Move, transport, positionLocation: San Francisco, CA, USA
| Salary: unspecified | Date posted: 11 Sep 2026
Bartender
Company: Arch Amenities Group |
with, discern, convey, express oneself, exchange information X See -- Detect, determine, perceive, identify, recognize, judgeLocation: San Francisco, CA, USA
| Salary: US$20 per hour | Date posted: 11 Sep 2026
Staff Software Engineer - Product Platform Engineering
Company: Pivotal Health |
instincts: you judge a system by whether it is useful and correct, and elegance comes second Able to engage in customerLocation: San Francisco, CA - Santa Monica, CA, USA
| Salary: US$22000 - 25000 per year | Date posted: 10 Sep 2026
Fitness Director
Company: Arch Amenities Group |
, perceive, identify, recognize, judge, observe, inspect, estimate, assess X Carry weight, lift -- Move, transport, positionLocation: San Francisco, CA, USA
| Salary: unspecified | Date posted: 09 Sep 2026
Paid Growth Marketing Expert
Company: Mercor |
reference answers and criteria distinguishing strong responses from plausible-but-wrong ones. Review and judge AI agentLocation: San Francisco, CA, USA
| Salary: unspecified | Date posted: 07 Sep 2026
Research Scientist, Apex Benchmarks
Company: Mercor |
standard for how we report results: confidence intervals, inter-rater agreement, human vs. model-as-judge calibration, held-out..., medicine or software engineering. Experience designing rubrics, model-as-judge pipelines, or human annotation programs