SOC 2 Readiness Assessment — attestly

Effective 2024-08-14 Source repository: Artificial425/attestly

This document is an internal readiness assessment, not a SOC 2 attestation report. A SOC 2 attestation can only be issued by a licensed CPA firm. Use this assessment to identify gaps, plan remediation, and brief your auditor before the engagement begins.

Service organization

Domenic Julian's workspace is the service organization in scope of this readiness assessment.

The attestly product is a B2B SaaS application designed to help companies with code-aware compliance. The system ingests customer source code, analyzes it for compliance and security issues, and generates relevant documentation. The system boundary includes the core application hosted on Railway and Fly.io, the serverless Postgres database managed by Neon, and integrations with third-party services for authentication (Clerk), payment processing (Stripe), email (Resend), and artificial intelligence model inference (Anthropic, Google Generative AI, OpenAI, and others).

Scope

Period. point-in-time as of 2024-08-14

Trust Services Criteria. Security (Common Criteria)

In scope.

• The attestly application, as provided to customers, identified by the repository Artificial425/attestly.

Out of scope.

• Corporate IT infrastructure
• Employee workstations and mobile devices
• Marketing website
• Third-party vendor services

Service commitments

• To protect the confidentiality and integrity of customer data processed by the attestly application.
• To ensure the attestly application is available for customer use in line with operational targets.
• To restrict logical access to the attestly application and its underlying infrastructure to authorized personnel.
• To implement and maintain security controls to protect against unauthorized access, use, or modification of customer data.

System requirements

• A formal risk management program is established to identify, assess, and mitigate threats to the system.
• Logical access controls are implemented to authenticate users and restrict access based on the principle of least privilege.
• A formal change management process is followed for all changes to the production environment.
• Security incidents are detected, responded to, and remediated in a timely manner.
• Third-party vendors are assessed for security and compliance risks before and during engagement.

Readiness snapshot

Metric	Value
Total controls	27
Implemented	2
Partially implemented	7
Planned	10
Not implemented	6
Not applicable	2
Readiness	8%

Process narratives

Risk assessment. A formal risk assessment process is planned but has not yet been completed. This process will involve identifying, analyzing, and evaluating risks to the attestly application and the data it processes. Attestly's continuous scanning provides ongoing identification of technical risks, such as software vulnerabilities and license compliance issues. The formal process will expand this to include operational, legal, and business risks, which will be tracked in a central risk register.

Change management. Changes to the attestly application are managed through source control in the Artificial425/attestly GitHub repository. All code modifications are submitted via pull requests, which require peer review before being merged into the main branch. This process ensures that changes are reviewed for correctness and adherence to coding standards. Attestly's drift detection provides an additional layer of oversight by alerting on significant changes to dependencies or data flows. A formal policy documenting testing and approval requirements is a planned enhancement.

Logical access. Logical access to the attestly application is managed by Clerk, which handles user authentication and credential management. Access to the underlying cloud infrastructure on Railway, Fly.io, and Neon is restricted to authorized engineering personnel and requires multi-factor authentication. Access is granted based on the principle of least privilege. Formal, periodic reviews of user access rights are planned to ensure that access levels remain appropriate.

System operations. The attestly application is hosted on Railway and Fly.io, which provide managed infrastructure and operational monitoring. The application database is a serverless Postgres instance managed by Neon, which handles automated backups and scaling. System health, performance, and availability are monitored through the dashboards and alerting features provided by these hosting platforms. A centralized logging and monitoring solution has not yet been implemented.

Continuous monitoring. The control environment is monitored on an ongoing basis through Attestly's automated tools, which provide drift alerts for changes in the technology stack, data flows, and dependencies. Management reviews these alerts and other operational metrics informally. A formal process for periodic management review of the overall effectiveness of the internal control system is a planned activity to ensure controls remain effective and aligned with business objectives.

Incident response. A formal incident response plan is currently under development. The plan will define roles and responsibilities, establish a severity classification matrix, and outline procedures for incident detection, containment, eradication, and recovery. It will also include a customer notification process with a commitment to notify affected customers of any confirmed data breach without undue delay, and in any case within 72 hours of confirmation.

Vendor management

Vendor management is a developing process. Attestly's continuous scanning identifies all third-party services and subprocessors integrated into the attestly product, providing a live inventory. This includes critical vendors such as Railway and Fly.io for hosting, Neon for data storage, Clerk for authentication, and multiple AI providers. A formal process for initial due diligence and ongoing performance and security review of these vendors is currently being established.

Inventory of record. The canonical list of subprocessors is maintained and published via the attestly public trust center.

Review cadence. Annual review for all critical vendors; ad-hoc review on Attestly drift alert.

Control matrix

Security (Common Criteria)

CC1.1 — Integrity and Ethical Values

Status: Planned

Intent. The organization demonstrates a commitment to integrity and ethical values by establishing standards of conduct and processes to ensure they are followed.

How we implement it. A formal code of conduct and information security policy have not yet been documented. Management is expected to lead by example, but there are no formal, documented policies that set the tone for the organization's commitment to security and ethical behavior.

Operator evidence required.

• A board- or management-approved Code of Conduct document.
• A board- or management-approved Information Security Policy.
• Evidence of policy distribution to all employees and contractors.

Gap. No formal, documented Code of Conduct or Information Security Policy exists. These documents are foundational for establishing a control environment.

CC1.2 — Board of Directors Oversight

Status: Planned

Intent. The board of directors or an equivalent oversight body demonstrates independence from management and exercises oversight for the development and performance of internal control.

How we implement it. Formal oversight structures and processes for internal control are not yet established. As the organization matures, a formal process for management to report on the effectiveness of the control environment to an oversight body will be implemented.

Operator evidence required.

• Minutes from board or management meetings where security and compliance are discussed.
• An organizational chart showing reporting lines for security-related roles.

Gap. There is no formal process for oversight of the internal control system. A regular cadence for management reporting on security posture to an oversight body needs to be established.

CC2.1 — Commitment to Competence

Status: Partially implemented

Intent. The organization is committed to attracting, developing, and retaining competent individuals in alignment with its objectives.

How we implement it. Hiring is conducted based on the technical and professional competence required for open roles. However, there is no formal process for defining job roles, responsibilities, and required competencies, particularly for security-sensitive positions.

Operator evidence required.

• Documented job descriptions for key roles, including security responsibilities.
• Records of background checks for employees in sensitive positions.

Gap. Job descriptions do not formally document security responsibilities, and a formal process for assessing competence for security roles is not in place.

CC2.2 — Accountability

Status: Planned

Intent. The organization holds individuals accountable for their internal control responsibilities in the pursuit of objectives.

How we implement it. Accountability for performance and internal control is handled informally. There are no formal performance reviews or documented processes that tie adherence to security policies to performance evaluation.

Operator evidence required.

• Employee handbook outlining responsibilities and accountability.
• Performance review templates that include security and compliance objectives.
• Disciplinary policy for security violations.

Gap. A formal system for holding individuals accountable for their security responsibilities has not been established. This includes performance management and disciplinary processes related to security.

CC2.3 — Security Awareness Training

Status: Not implemented

Intent. The organization provides security awareness training to all personnel to ensure they understand their responsibilities.

How we implement it. There is currently no formal security awareness training program for employees or contractors. Personnel are expected to be aware of security best practices, but this is not verified or reinforced through a structured training curriculum.

Operator evidence required.

• Security awareness training materials or subscription records.
• Training completion records for all employees and contractors.
• Signed confidentiality agreements from all personnel.

Gap. No security awareness training program is in place. Personnel may be unaware of current threats and their specific responsibilities for protecting company and customer data.

CC3.1 — Risk Assessment

Status: Planned

Intent. The organization specifies objectives with sufficient clarity to enable the identification and assessment of risks relating to objectives.

How we implement it. While the organization has business objectives, these have not been formally translated into specific, measurable objectives for the purpose of a formal risk assessment. A comprehensive risk assessment process has not yet been conducted.

Operator evidence required.

• Documented risk assessment methodology.
• A risk register detailing identified risks, analysis, and response plans.

Gap. A formal risk assessment process has not been implemented. This is a critical gap for systematically identifying and managing threats to the service.

CC3.2 — Vendor Risk Management

Status: Partially implemented

Intent. The organization identifies and assesses risks associated with vendors and business partners.

How we implement it. The attestly application relies on several third-party subprocessors, including Clerk, Neon, Stripe, Fly.io, Railway, and multiple AI providers. Attestly's scanning provides an inventory of these vendors. However, a formal risk assessment and due diligence process for selecting and reviewing these vendors is not yet documented.

Attestly evidence.

• Live subprocessor list generated by Attestly scans.

Operator evidence required.

• A documented vendor management policy.
• Completed security questionnaires or due diligence records for critical vendors.
• Vendor agreements with required security and privacy clauses.

Gap. While vendors are identified, there is no formal, documented process for assessing their security posture or for ongoing monitoring.

CC3.3 — Risk Identification and Analysis

Status: Planned

Intent. The organization identifies risks to the achievement of its objectives across the entity and analyzes risks as a basis for determining how the risks should be managed.

How we implement it. Risk identification is currently ad-hoc, driven by operational issues or technical findings. A systematic process to identify and analyze risks from internal and external sources, including potential for fraud, has not been implemented.

Attestly evidence.

• Drift alert notifications for changes in the technology stack.

Operator evidence required.

• Risk assessment register with detailed risk analysis.
• Minutes from risk assessment workshops or meetings.

Gap. A formal, repeatable process for risk identification and analysis is not in place.

CC3.4 — Fraud Risk Assessment

Status: Planned

Intent. The organization considers the potential for fraud in assessing risks to the achievement of objectives.

How we implement it. A specific fraud risk assessment has not been conducted. While payment processing is handled by Stripe, which has its own fraud detection, internal fraud risks related to data access and system administration have not been formally assessed.

Operator evidence required.

• A documented fraud risk assessment.
• Controls designed to mitigate identified fraud risks.

Gap. No formal fraud risk assessment has been performed to consider how fraud could impact the service's objectives.

CC4.1 — Control Selection and Development

Status: Partially implemented

Intent. The organization selects and develops control activities that contribute to the mitigation of risks to the achievement of objectives to acceptable levels.

How we implement it. Control activities are being implemented, such as using Clerk for authentication and Neon for database management. However, the selection and development of these controls are not yet tied to a formal risk assessment. The current set of controls has been developed based on industry best practices rather than a specific risk analysis.

Operator evidence required.

• A control catalog mapping controls to identified risks.
• System design documents that specify security controls.

Gap. Controls are not yet systematically selected and designed based on the output of a formal risk assessment.

CC4.2 — Technology Controls

Status: Partially implemented

Intent. The organization selects and develops general control activities over technology to support the achievement of objectives.

How we implement it. General technology controls are in place through the use of managed services like Railway, Fly.io, and Neon, which are responsible for the underlying infrastructure security. However, a comprehensive set of policies and procedures governing the organization's own technology infrastructure and development lifecycle is not yet documented.

Operator evidence required.

• Network diagrams and data flow diagrams.
• Configuration standards for production systems.
• A formal secure software development lifecycle (SDLC) policy.

Gap. Policies and procedures for technology controls, such as a secure SDLC policy and configuration standards, are not formally documented.

CC5.1 — Use of Information and Communication

Status: Partially implemented

Intent. The organization obtains or generates and uses relevant, quality information to support the functioning of internal control.

How we implement it. Information from production systems is used for operational monitoring. Hosting providers Railway and Fly.io provide logs and metrics. However, there is no formal process for defining information requirements for control monitoring or for assessing the quality and relevance of this information.

Attestly evidence.

• Audit log of repository scans and findings.

Operator evidence required.

• A list of information sources used for monitoring controls.
• Reports or dashboards used by management to oversee the control environment.

Gap. The process for identifying, capturing, and using information to monitor the effectiveness of internal controls is informal.

CC5.2 — Internal Communication

Status: Planned

Intent. The organization internally communicates information, including objectives and responsibilities for internal control, necessary to support the functioning of internal control.

How we implement it. Internal communication regarding security responsibilities is informal. There are no dedicated channels, policies, or regular meetings established for communicating internal control information to all relevant personnel.

Operator evidence required.

• An internal communication policy.
• Examples of internal communications regarding security (e.g., emails, meeting minutes).
• A process for personnel to report security concerns or policy exceptions.

Gap. No formal mechanisms exist for communicating internal control objectives and responsibilities throughout the organization.

CC5.3 — External Communication

Status: Planned

Intent. The organization communicates with external parties regarding matters affecting the functioning of internal control.

How we implement it. A process for communicating with customers and other external parties about security matters, such as incidents or changes to security practices, is not yet formalized. The organization's contact email is admin@attestly.dev, but formal policies and procedures for external communication are not in place.

Operator evidence required.

• A public-facing security or trust page.
• Customer notification templates for security incidents.
• A policy for responding to external inquiries about security.

Gap. Formal policies and procedures for communicating with external parties on security matters are not documented.

CC6.1 — Ongoing and/or Separate Evaluations

Status: Partially implemented

Intent. The organization selects, develops, and performs ongoing and/or separate evaluations to ascertain whether the components of internal control are present and functioning.

How we implement it. Ongoing evaluations are performed through Attestly's continuous scanning and drift detection, which monitor the state of the codebase and its dependencies. However, separate, periodic evaluations, such as internal audits or penetration tests, are not yet being performed.

Attestly evidence.

• Drift alert notifications.
• Audit log of continuous scanning activities.

Operator evidence required.

• An internal audit plan or charter.
• Results of the most recent penetration test.
• Management's review of monitoring activities.

Gap. While continuous monitoring is in place via Attestly, periodic, independent evaluations like penetration tests or internal audits are not yet conducted.

CC6.2 — Evaluation and Communication of Deficiencies

Status: Planned

Intent. The organization evaluates and communicates internal control deficiencies in a timely manner to those parties responsible for taking corrective action, including senior management and the board of directors, as appropriate.

How we implement it. A formal process for evaluating, tracking, and communicating control deficiencies has not been established. Findings from Attestly scans are available to developers, but there is no formal process for escalating significant deficiencies to management or tracking them to remediation.

Attestly evidence.

• Attestly findings dashboard.

Operator evidence required.

• A policy for identifying and reporting control deficiencies.
• A remediation tracking log or system.
• Reports to management on the status of open deficiencies.

Gap. There is no formal process for the evaluation, reporting, and remediation tracking of internal control deficiencies.

CC7.1 — Logical Access Security

Status: Implemented

Intent. The organization implements logical access security software, infrastructure, and architectures over protected information assets to protect them from security events to meet the entity’s objectives.

How we implement it. Logical access to the attestly application is managed by Clerk, which provides user authentication services. Access to underlying infrastructure (Railway, Fly.io, Neon) is managed through the respective providers' IAM systems. This establishes a baseline for logical access control.

Operator evidence required.

• Screenshots of Clerk configuration showing password policy and MFA enforcement.
• A list of authorized users for infrastructure platforms.
• Role-based access control (RBAC) configuration.

CC7.2 — Identity and Access Management

Status: Partially implemented

Intent. Prior to issuing system credentials and granting system access, the entity registers and authorizes new internal and external users whose access is administered by the entity. For those users whose access is administered by the entity, user system credentials are removed when user access is no longer authorized.

How we implement it. User registration for the application is handled by Clerk. For internal personnel, access to third-party systems is granted as needed. However, there is no formal, documented process for user access requests, approvals, and de-provisioning upon termination or role change. Access reviews are not performed.

Operator evidence required.

• A documented user access control policy.
• Records of access request and approval for a sample of users.
• Evidence of timely de-provisioning for a sample of terminated employees.
• Records of periodic user access reviews.

Gap. Formal processes for authorizing, reviewing, and de-provisioning user access are not documented or consistently followed.

CC7.3 — Authentication

Status: Implemented

Intent. The entity authorizes, modifies, or removes access to protected information and related assets based on authentication of the user’s identity in accordance with the entity’s objectives.

How we implement it. Authentication for the attestly application is enforced by Clerk. This service manages user credentials and authentication flows. Access to infrastructure requires authentication through the respective cloud provider's login system.

Operator evidence required.

• Documentation of authentication mechanisms in use.
• Password policy configuration (length, complexity, history).
• MFA configuration and enforcement policy.

CC7.4 — Physical Access

Status: Not applicable

Intent. The entity limits physical access to facilities and protected information assets (for example, data center facilities) to authorized individuals to meet the entity’s objectives.

How we implement it. The organization does not operate its own data centers. The attestly application and its data are hosted on cloud infrastructure provided by Railway, Fly.io, and Neon. These providers are responsible for physical security of the data centers. Their compliance reports (e.g., SOC 2) provide assurance over these controls.

Operator evidence required.

• SOC 2 or equivalent reports from Railway, Fly.io, and Neon.

CC7.5 — Data Disposal

Status: Not applicable

Intent. The entity protects against the unauthorized removal of protected information and system components by implementing controls to meet the entity’s objectives.

How we implement it. The organization does not manage physical media. Data disposal is managed by the cloud hosting providers (Railway, Fly.io, Neon) according to their internal procedures. The organization relies on the contractual agreements and compliance reports of its providers for assurance over media sanitization and disposal.

Operator evidence required.

• SOC 2 or equivalent reports from Railway, Fly.io, and Neon detailing their media disposal procedures.

CC8.1 — Change Management

Status: Partially implemented

Intent. The entity authorizes, designs, develops or acquires, configures, documents, tests, approves, and implements changes to infrastructure, data, software, and procedures to meet its objectives.

How we implement it. Changes to the attestly application code are managed via pull requests in the Artificial425/attestly GitHub repository. This provides a mechanism for code review. However, a formal change management policy documenting requirements for testing, approval, and emergency changes is not in place.

Attestly evidence.

• Audit log of pull request history from GitHub.

Operator evidence required.

• A documented change management policy.
• Evidence of testing and approval for a sample of changes.
• A process for emergency changes.

Gap. The change management process is informal and lacks formal documentation, testing, and approval requirements.

CC9.1 — Risk Mitigation

Status: Partially implemented

Intent. The entity identifies, selects, and develops risk mitigation activities for risks affecting the system.

How we implement it. Risk mitigation activities are in place, such as using managed services for hosting and authentication, which transfers some operational risk. However, these activities are not part of a comprehensive risk management program where mitigation strategies are selected and implemented based on a formal risk assessment.

Operator evidence required.

• A risk treatment plan.
• Business continuity and disaster recovery plans.
• Evidence of security controls implemented to mitigate specific risks (e.g., firewall rules, encryption).

Gap. Risk mitigation is ad-hoc rather than being driven by a formal risk assessment and treatment plan.

CC9.2 — Incident Management

Status: Not implemented

Intent. The entity identifies, develops, and implements activities to respond to and recover from security events.

How we implement it. There is no formal, documented incident response plan. While the team would respond to a security event, there are no predefined procedures for detection, containment, eradication, recovery, and post-incident analysis. Customer notification procedures and SLAs are also not defined.

Operator evidence required.

• A documented Incident Response Plan.
• Records of incident response tests or tabletop exercises.
• An incident tracking log or system.

Gap. A formal incident response plan does not exist. This prevents a coordinated and effective response to security incidents.

Top gaps to close

Priority	Criterion	Summary	Recommended action
High	CC9.2	No formal Incident Response Plan exists.	Develop, approve, and test a comprehensive Incident Response Plan that includes procedures for detection, containment, eradication, recovery, and customer notification.
High	CC3.1	A formal, documented risk assessment has not been conducted.	Implement a formal risk assessment methodology and conduct an organization-wide risk assessment. Document the results in a risk register and create a risk treatment plan.
High	CC1.1	Core governance documents like an Information Security Policy and Code of Conduct are missing.	Draft, approve, and distribute a formal Information Security Policy and Code of Conduct to all personnel.
Medium	CC2.3	No security awareness training program is in place for personnel.	Implement a mandatory security awareness training program for all new and existing employees and contractors. Track completion and conduct annual refreshers.
Medium	CC7.2	User access review and formal de-provisioning processes are not documented.	Document and implement a formal user access control policy that includes procedures for access requests, approvals, periodic reviews, and timely de-provisioning.

Next steps

1. Develop and approve foundational governance documents, starting with an Information Security Policy and an Incident Response Plan.
2. Conduct a formal, organization-wide risk assessment to identify and prioritize risks, and create a risk treatment plan.
3. Implement a security awareness training program for all employees and contractors.
4. Formalize and document the change management and user access control processes, including requirements for testing, approval, and periodic access reviews.
5. Engage a third-party firm to conduct a penetration test of the attestly application.
6. Collect and organize all required evidence in preparation for a formal SOC 2 Type 1 audit.

Auditor evidence pack

Source	Artefact	How to obtain
Operator-supplied	Information Security Policy	To be created, approved by management, and distributed to all personnel.
Operator-supplied	Incident Response Plan	To be created, approved by management, and tested via a tabletop exercise.
Operator-supplied	Risk Assessment Register	To be created as the output of the formal risk assessment process.
Attestly artefact	Subprocessor Inventory and Due Diligence	The inventory is generated from the live subprocessor list in the Attestly trust center. Due diligence evidence (e.g., vendor SOC 2 reports) must be collected by the operator.
Operator-supplied	Authentication Provider Configuration	Provide screenshots from the Clerk administrative console showing MFA enforcement and password policies.
Attestly artefact	Change Management Evidence	Export a sample of pull request histories from the Artificial425/attestly GitHub repository, showing code review and approval.
External attestation	Cloud Provider Compliance Reports	Download the latest SOC 2 Type 2 reports for Railway, Fly.io, and Neon from their respective trust centers or customer portals.